[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:13:48 dist-test-slave-fd7l krb5kdc[21729](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:13:48 dist-test-slave-fd7l krb5kdc[21729](info): set up 2 sockets
Aug 11 17:13:48 dist-test-slave-fd7l krb5kdc[21729](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 17:13:49.307889 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.000s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:13:49 dist-test-slave-fd7l krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:13:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:13:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:13:49.457288 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:44183
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:44183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:13:49.597767 21742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:13:49.598057 21742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:13:49.598140 21742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:13:49.602324 21742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:13:49.602396 21742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:13:49.602416 21742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:13:49.602433 21742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:13:49.602449 21742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:13:49.608269 21742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:44183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:44183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:13:49.610303 21742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:13:49.611712 21742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:13:49.630788 21751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:49.631042 21748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:49.631206 21749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:49.630954 21742 server_base.cc:1034] running on GCE node
I20260811 17:13:49.632547 21742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:13:49.635411 21742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:13:49.638720 21742 hybrid_clock.cc:648] HybridClock initialized: now 1786468429638647 us; error 97 us; skew 500 ppm
Aug 11 17:13:49 dist-test-slave-fd7l krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468429, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:13:49.643079 21742 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:13:49.644574 21742 webserver.cc:460] Webserver started at http://127.21.54.190:39041/ using document root <none> and password file <none>
I20260811 17:13:49.645584 21742 fs_manager.cc:359] Metadata directory not provided
I20260811 17:13:49.645699 21742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:13:49.646212 21742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:13:49.649227 21742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "3c7402c50d8549b9846b74ef28d0c65c"
format_stamp: "Formatted at 2026-08-11 17:13:49 on dist-test-slave-fd7l"
server_key: "e1bb216cad0bdffa08895b9706bfffb4"
server_key_iv: "b51ed856f95220c0ab93348d9cb721f0"
server_key_version: "encryptionkey@0"
I20260811 17:13:49.650134 21742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "3c7402c50d8549b9846b74ef28d0c65c"
format_stamp: "Formatted at 2026-08-11 17:13:49 on dist-test-slave-fd7l"
server_key: "e1bb216cad0bdffa08895b9706bfffb4"
server_key_iv: "b51ed856f95220c0ab93348d9cb721f0"
server_key_version: "encryptionkey@0"
I20260811 17:13:49.655521 21742 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 17:13:49.660871 21758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:49.662602 21742 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260811 17:13:49.662859 21742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "3c7402c50d8549b9846b74ef28d0c65c"
format_stamp: "Formatted at 2026-08-11 17:13:49 on dist-test-slave-fd7l"
server_key: "e1bb216cad0bdffa08895b9706bfffb4"
server_key_iv: "b51ed856f95220c0ab93348d9cb721f0"
server_key_version: "encryptionkey@0"
I20260811 17:13:49.663013 21742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:13:49.676074 21742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:13:49.681495 21742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:13:49.681739 21742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:13:49.694034 21742 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:44183
I20260811 17:13:49.694077 21820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:44183 every 8 connection(s)
I20260811 17:13:49.695538 21742 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:13:49.697696 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 21742
I20260811 17:13:49.697904 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:13:49.699456 21722 external_mini_cluster.cc:1428] Setting key cb910b468721f5d022a371bd2c95d59e
I20260811 17:13:49.702252 21822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:49.714409 21822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c: Bootstrap starting.
I20260811 17:13:49.719350 21822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:49.721024 21822 log.cc:824] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c: Log is configured to *not* fsync() on all Append() calls
I20260811 17:13:49.725095 21822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c: No bootstrap required, opened a new log
Aug 11 17:13:49 dist-test-slave-fd7l krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:13:49.730428 21822 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7402c50d8549b9846b74ef28d0c65c" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44183 } }
I20260811 17:13:49.731011 21822 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:49.731149 21822 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c7402c50d8549b9846b74ef28d0c65c, State: Initialized, Role: FOLLOWER
I20260811 17:13:49.732419 21822 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7402c50d8549b9846b74ef28d0c65c" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44183 } }
I20260811 17:13:49.732853 21822 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:13:49.733009 21822 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:13:49.733196 21822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:49.734320 21822 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7402c50d8549b9846b74ef28d0c65c" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44183 } }
I20260811 17:13:49.735023 21822 leader_election.cc:302] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c7402c50d8549b9846b74ef28d0c65c; no voters:
I20260811 17:13:49.735704 21822 leader_election.cc:288] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:13:49.735956 21827 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:13:49.736260 21827 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c [term 1 LEADER]: Becoming Leader. State: Replica: 3c7402c50d8549b9846b74ef28d0c65c, State: Running, Role: LEADER
I20260811 17:13:49.736902 21827 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7402c50d8549b9846b74ef28d0c65c" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44183 } }
I20260811 17:13:49.737282 21825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:49.707217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50364 (local address 127.21.54.190:44183)
0811 17:13:49.707716 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:49.707746 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:49.707821 (+ 75us) server_negotiation.cc:407] Connection header received
0811 17:13:49.712332 (+ 4511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:49.712352 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:49.712893 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:49.713201 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:49.717541 (+ 4340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:49.718659 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:49.720938 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:49.721588 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:49.721817 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:49.727613 (+ 5796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:49.727671 (+ 58us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:49.727691 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:49.727733 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:49.732592 (+ 4859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:49.733417 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:49.733424 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:49.733428 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:49.733613 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:49.734021 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:49.734024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:49.734026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:49.734657 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:49.734893 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:49.735682 (+ 789us) server_negotiation.cc:299] Negotiation successful
0811 17:13:49.735994 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":167,"threads_started":1}
I20260811 17:13:49.737852 21822 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:13:49.741904 21829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c7402c50d8549b9846b74ef28d0c65c. Latest consensus state: current_term: 1 leader_uuid: "3c7402c50d8549b9846b74ef28d0c65c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7402c50d8549b9846b74ef28d0c65c" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44183 } } }
I20260811 17:13:49.742084 21829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c [sys.catalog]: This master's current role is: LEADER
I20260811 17:13:49.742725 21846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:13:49.742724 21828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c7402c50d8549b9846b74ef28d0c65c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7402c50d8549b9846b74ef28d0c65c" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44183 } } }
I20260811 17:13:49.742861 21828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c [sys.catalog]: This master's current role is: LEADER
I20260811 17:13:49.747339 21846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:13:49.755801 21846 catalog_manager.cc:1347] Generated new cluster ID: 8dcf3e07b16749f88c1117d4b044fa37
I20260811 17:13:49.755905 21846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:13:49.773438 21846 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:13:49.774899 21846 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:13:49.784758 21846 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c7402c50d8549b9846b74ef28d0c65c: Generated new TSK 0
I20260811 17:13:49.785801 21846 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
I20260811 17:13:49.825549 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44183
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:13:49.950404 21852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:13:49.950731 21852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:13:49.950779 21852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:13:49.955130 21852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:13:49.955210 21852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:13:49.955276 21852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:13:49.960950 21852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44183
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:13:49.962221 21852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:13:49.963435 21852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:13:49.974429 21861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:49.974478 21858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:49.974440 21859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:49.975008 21852 server_base.cc:1034] running on GCE node
I20260811 17:13:49.975443 21852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:13:49.976141 21852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:13:49.977378 21852 hybrid_clock.cc:648] HybridClock initialized: now 1786468429977351 us; error 45 us; skew 500 ppm
Aug 11 17:13:49 dist-test-slave-fd7l krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468429, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:13:49.981060 21852 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:13:49.982156 21852 webserver.cc:460] Webserver started at http://127.21.54.129:34317/ using document root <none> and password file <none>
I20260811 17:13:49.982820 21852 fs_manager.cc:359] Metadata directory not provided
I20260811 17:13:49.982924 21852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:13:49.983253 21852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:13:49.985725 21852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "110ae32a5069450c8d19e1eacb09111b"
format_stamp: "Formatted at 2026-08-11 17:13:49 on dist-test-slave-fd7l"
server_key: "1ea8fc5750937235c9c14b145f1c4446"
server_key_iv: "13a4ddf9c7fdfc45547cf09ead55686b"
server_key_version: "encryptionkey@0"
I20260811 17:13:49.986385 21852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "110ae32a5069450c8d19e1eacb09111b"
format_stamp: "Formatted at 2026-08-11 17:13:49 on dist-test-slave-fd7l"
server_key: "1ea8fc5750937235c9c14b145f1c4446"
server_key_iv: "13a4ddf9c7fdfc45547cf09ead55686b"
server_key_version: "encryptionkey@0"
I20260811 17:13:49.990828 21852 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:13:49.993872 21870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:49.995374 21852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:13:49.995534 21852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "110ae32a5069450c8d19e1eacb09111b"
format_stamp: "Formatted at 2026-08-11 17:13:49 on dist-test-slave-fd7l"
server_key: "1ea8fc5750937235c9c14b145f1c4446"
server_key_iv: "13a4ddf9c7fdfc45547cf09ead55686b"
server_key_version: "encryptionkey@0"
I20260811 17:13:49.995692 21852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:13:50.016577 21852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:13:50.020485 21852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:13:50.020880 21852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:13:50.021692 21852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:13:50.023087 21852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:13:50.023175 21852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:50.023259 21852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:13:50.023319 21852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:50.035782 21852 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:45181
I20260811 17:13:50.035808 21997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:45181 every 8 connection(s)
I20260811 17:13:50.036826 21852 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:13:50.044353 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 21852
I20260811 17:13:50.044502 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:13:50.044821 21722 external_mini_cluster.cc:1428] Setting key 3482d67d7ab9581fe3eb613e75366e6c
Aug 11 17:13:50 dist-test-slave-fd7l krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468429, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:13:50.054245 21825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:50.039412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:40163 (local address 127.21.54.190:44183)
0811 17:13:50.039576 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:50.039580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:50.041171 (+ 1591us) server_negotiation.cc:407] Connection header received
0811 17:13:50.042229 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:50.042233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:50.042297 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:50.042424 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:50.043778 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:50.044415 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:50.046087 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:50.046411 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:50.046518 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:50.049586 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:50.049592 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:50.049598 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:50.049636 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:50.051339 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:50.052224 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:50.052227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:50.052229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:50.052314 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:50.052867 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:50.052870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:50.052872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:50.053098 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:50.053180 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:50.053979 (+ 799us) server_negotiation.cc:299] Negotiation successful
0811 17:13:50.054108 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 17:13:50.055701 22001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:50.039807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:44183 (local address 127.21.54.129:40163)
0811 17:13:50.040984 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:50.041042 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:50.041953 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:50.042640 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:50.042647 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:50.043067 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:50.043607 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:50.043623 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:50.044601 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:50.044608 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:50.045927 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:50.045939 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:50.046602 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:50.046607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:50.046825 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:50.047544 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:50.047563 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:50.049368 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:50.051595 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:50.051611 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:50.051633 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:50.052078 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:50.052431 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:50.052438 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:50.052443 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:50.052717 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:50.053245 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:50.053255 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:50.053670 (+ 415us) client_negotiation.cc:769] Sending connection context
0811 17:13:50.053929 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 17:13:50.054374 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":970,"thread_start_us":775,"threads_started":1}
I20260811 17:13:50.057247 21999 heartbeater.cc:347] Connected to a master server at 127.21.54.190:44183
I20260811 17:13:50.057561 21999 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.21.54.130@KRBTEST.COM; defaulting to no policy
I20260811 17:13:50.058228 21999 heartbeater.cc:511] Master 127.21.54.190:44183 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.130@KRBTEST.COM" created.
I20260811 17:13:50.060936 21776 ts_manager.cc:194] Registered new tserver with Master: 110ae32a5069450c8d19e1eacb09111b (127.21.54.129:45181)
I20260811 17:13:50.062685 21776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:40163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
I20260811 17:13:50.080442 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44183
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:13:50.208068 22004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:13:50.208367 22004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:13:50.208413 22004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:13:50.212934 22004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:13:50.213042 22004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:13:50.213110 22004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:13:50.219023 22004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44183
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:13:50.220186 22004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:13:50.221249 22004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:13:50.231806 22011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:50.231825 22010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:50.231825 22015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:50.231990 22004 server_base.cc:1034] running on GCE node
I20260811 17:13:50.232492 22004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:13:50.233302 22004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:13:50.234513 22004 hybrid_clock.cc:648] HybridClock initialized: now 1786468430234473 us; error 50 us; skew 500 ppm
Aug 11 17:13:50 dist-test-slave-fd7l krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468430, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:13:50.237742 22004 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:13:50.238852 22004 webserver.cc:460] Webserver started at http://127.21.54.130:40191/ using document root <none> and password file <none>
I20260811 17:13:50.239488 22004 fs_manager.cc:359] Metadata directory not provided
I20260811 17:13:50.239562 22004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:13:50.239796 22004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:13:50.242090 22004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "462483e487114fa69a658587211edeee"
format_stamp: "Formatted at 2026-08-11 17:13:50 on dist-test-slave-fd7l"
server_key: "2f4d82cee6c83cb48dbc722b92381412"
server_key_iv: "38af9c187366d437fdb33744b88e5609"
server_key_version: "encryptionkey@0"
I20260811 17:13:50.242707 22004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "462483e487114fa69a658587211edeee"
format_stamp: "Formatted at 2026-08-11 17:13:50 on dist-test-slave-fd7l"
server_key: "2f4d82cee6c83cb48dbc722b92381412"
server_key_iv: "38af9c187366d437fdb33744b88e5609"
server_key_version: "encryptionkey@0"
I20260811 17:13:50.247130 22004 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:13:50.250049 22023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:50.251590 22004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:13:50.251739 22004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "462483e487114fa69a658587211edeee"
format_stamp: "Formatted at 2026-08-11 17:13:50 on dist-test-slave-fd7l"
server_key: "2f4d82cee6c83cb48dbc722b92381412"
server_key_iv: "38af9c187366d437fdb33744b88e5609"
server_key_version: "encryptionkey@0"
I20260811 17:13:50.251883 22004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:13:50.266767 22004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:13:50.270762 22004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:13:50.270998 22004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:13:50.271708 22004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:13:50.272696 22004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:13:50.272747 22004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:50.272821 22004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:13:50.272856 22004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:50.283555 22004 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:39193
I20260811 17:13:50.283710 22156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:39193 every 8 connection(s)
I20260811 17:13:50.284783 22004 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:13:50.289011 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 22004
I20260811 17:13:50.289142 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:13:50.289430 21722 external_mini_cluster.cc:1428] Setting key 0567a8e4cce2169ea7965801b8123e38
Aug 11 17:13:50 dist-test-slave-fd7l krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468430, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.54.131@KRBTEST.COM; defaulting to no policy
I20260811 17:13:50.302883 21825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:50.286984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:38701 (local address 127.21.54.190:44183)
0811 17:13:50.287140 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:50.287146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:50.288386 (+ 1240us) server_negotiation.cc:407] Connection header received
0811 17:13:50.289886 (+ 1500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:50.289893 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:50.289969 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:50.290086 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:50.291817 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:50.292369 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:50.294189 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:50.294582 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:50.294756 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:50.297986 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:50.297993 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:50.298000 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:50.298041 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:50.300109 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:50.300890 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:50.300896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:50.300898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:50.301012 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:50.301411 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:50.301414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:50.301418 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:50.301698 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:50.301815 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:50.302480 (+ 665us) server_negotiation.cc:299] Negotiation successful
0811 17:13:50.302744 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 17:13:50.303674 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:50.287389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:44183 (local address 127.21.54.130:38701)
0811 17:13:50.288218 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:50.288265 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:50.289541 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:50.290333 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:50.290343 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:50.290815 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:50.291615 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:50.291652 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:50.292556 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:50.292563 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:50.294040 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:50.294052 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:50.294934 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:50.294939 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:50.295108 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:50.295829 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:50.295848 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:50.297769 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:50.300251 (+ 2482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:50.300257 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:50.300333 (+ 76us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:50.300754 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:50.301132 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:50.301134 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:50.301136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:50.301299 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:50.301810 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:50.301816 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:50.302129 (+ 313us) client_negotiation.cc:769] Sending connection context
0811 17:13:50.302316 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 17:13:50.302611 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":118,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.131@KRBTEST.COM" created.
I20260811 17:13:50.305249 22157 heartbeater.cc:347] Connected to a master server at 127.21.54.190:44183
I20260811 17:13:50.305553 22157 heartbeater.cc:464] Registering TS with master...
I20260811 17:13:50.306121 22157 heartbeater.cc:511] Master 127.21.54.190:44183 requested a full tablet report, sending...
I20260811 17:13:50.307365 21776 ts_manager.cc:194] Registered new tserver with Master: 462483e487114fa69a658587211edeee (127.21.54.130:39193)
I20260811 17:13:50.308003 21776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:38701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
I20260811 17:13:50.325183 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44183
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:13:50.451864 22163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:13:50.452210 22163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:13:50.452286 22163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:13:50.456629 22163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:13:50.456703 22163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:13:50.456802 22163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:13:50.462531 22163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44183
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:13:50.463737 22163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:13:50.464861 22163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:13:50.475423 22163 server_base.cc:1034] running on GCE node
W20260811 17:13:50.475356 22170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:50.475361 22169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:50.475361 22173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:50.476116 22163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:13:50.476850 22163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:13:50.478024 22163 hybrid_clock.cc:648] HybridClock initialized: now 1786468430478015 us; error 38 us; skew 500 ppm
Aug 11 17:13:50 dist-test-slave-fd7l krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468430, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:13:50.481576 22163 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:13:50.482638 22163 webserver.cc:460] Webserver started at http://127.21.54.131:41999/ using document root <none> and password file <none>
I20260811 17:13:50.483258 22163 fs_manager.cc:359] Metadata directory not provided
I20260811 17:13:50.483321 22163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:13:50.483554 22163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:13:50.485816 22163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "4986126d741d429ba72c7fd00a156e08"
format_stamp: "Formatted at 2026-08-11 17:13:50 on dist-test-slave-fd7l"
server_key: "df4580a560c0777a9e36b121dba0ddee"
server_key_iv: "e964a6675b7a6f6299a78d6439a09271"
server_key_version: "encryptionkey@0"
I20260811 17:13:50.486419 22163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "4986126d741d429ba72c7fd00a156e08"
format_stamp: "Formatted at 2026-08-11 17:13:50 on dist-test-slave-fd7l"
server_key: "df4580a560c0777a9e36b121dba0ddee"
server_key_iv: "e964a6675b7a6f6299a78d6439a09271"
server_key_version: "encryptionkey@0"
I20260811 17:13:50.490442 22163 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:13:50.493185 22181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:50.494426 22163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:13:50.494580 22163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "4986126d741d429ba72c7fd00a156e08"
format_stamp: "Formatted at 2026-08-11 17:13:50 on dist-test-slave-fd7l"
server_key: "df4580a560c0777a9e36b121dba0ddee"
server_key_iv: "e964a6675b7a6f6299a78d6439a09271"
server_key_version: "encryptionkey@0"
I20260811 17:13:50.494709 22163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:13:50.505757 22163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:13:50.508927 22163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:13:50.509150 22163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:13:50.509794 22163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:13:50.510732 22163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:13:50.510782 22163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:50.510847 22163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:13:50.510880 22163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:50.520318 22163 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:38569
I20260811 17:13:50.520340 22307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:38569 every 8 connection(s)
I20260811 17:13:50.521416 22163 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:13:50.523756 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 22163
I20260811 17:13:50.523888 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:13:50.524206 21722 external_mini_cluster.cc:1428] Setting key f56faa8f4aea5d50b41c9b0bf18af7c4
Aug 11 17:13:50 dist-test-slave-fd7l krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468430, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:13:50.537873 21825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:50.523816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:49393 (local address 127.21.54.190:44183)
0811 17:13:50.524015 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:50.524021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:50.525832 (+ 1811us) server_negotiation.cc:407] Connection header received
0811 17:13:50.526897 (+ 1065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:50.526900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:50.526956 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:50.527049 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:50.528459 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:50.529056 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:50.530245 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:50.530471 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:50.530569 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:50.533481 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:50.533484 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:50.533486 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:50.533562 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:50.535111 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:50.535811 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:50.535816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:50.535819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:50.535972 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:50.536454 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:50.536458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:50.536459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:50.536772 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:50.536906 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:50.537514 (+ 608us) server_negotiation.cc:299] Negotiation successful
0811 17:13:50.537674 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 17:13:50.538980 22311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:50.524340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:44183 (local address 127.21.54.131:49393)
0811 17:13:50.525645 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:50.525710 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:50.526670 (+ 960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:50.527371 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:50.527379 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:50.527790 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:50.528318 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:50.528332 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:50.529183 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:50.529186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:50.530088 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:50.530098 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:50.530657 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:50.530660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:50.530828 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:50.531466 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:50.531483 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:50.533270 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:50.535252 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:50.535258 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:50.535274 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:50.535649 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:50.536105 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:50.536109 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:50.536111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:50.536308 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:50.536908 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:50.536919 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:50.537221 (+ 302us) client_negotiation.cc:769] Sending connection context
0811 17:13:50.537483 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 17:13:50.537893 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1039,"thread_start_us":854,"threads_started":1}
I20260811 17:13:50.540354 22309 heartbeater.cc:347] Connected to a master server at 127.21.54.190:44183
I20260811 17:13:50.540668 22309 heartbeater.cc:464] Registering TS with master...
I20260811 17:13:50.541175 22309 heartbeater.cc:511] Master 127.21.54.190:44183 requested a full tablet report, sending...
I20260811 17:13:50.542272 21776 ts_manager.cc:194] Registered new tserver with Master: 4986126d741d429ba72c7fd00a156e08 (127.21.54.131:38569)
I20260811 17:13:50.542830 21776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:49393
I20260811 17:13:50.552472 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 17:13:50 dist-test-slave-fd7l krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468430, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 17:13:50 dist-test-slave-fd7l krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468430, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.54.129@KRBTEST.COM
I20260811 17:13:50.584498 22317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:50.572844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58030 (local address 127.21.54.129:45181)
0811 17:13:50.573141 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:50.573147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:50.573424 (+ 277us) server_negotiation.cc:407] Connection header received
0811 17:13:50.573505 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:50.573512 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:50.573729 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:50.573883 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:50.574636 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:50.575567 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:50.576608 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:50.577076 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:50.577180 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:50.580204 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:50.580218 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:50.580237 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:50.580272 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:50.582235 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:50.582746 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:50.582753 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:50.582754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:50.582849 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:50.583259 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:50.583262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:50.583263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:50.583613 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:50.583799 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:50.584078 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 17:13:50.584282 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":85,"threads_started":1}
W20260811 17:13:50.585846 21899 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58030
Aug 11 17:13:50 dist-test-slave-fd7l krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 17:13:50 dist-test-slave-fd7l krb5kdc[21729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.129@KRBTEST.COM
I20260811 17:13:50.616565 22317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:50.607113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58038 (local address 127.21.54.129:45181)
0811 17:13:50.607283 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:50.607288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:50.607413 (+ 125us) server_negotiation.cc:407] Connection header received
0811 17:13:50.607645 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:50.607647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:50.607697 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:50.607804 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:50.608596 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:50.609123 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:50.609990 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:50.610222 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:50.610355 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:50.613006 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:50.613009 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:50.613011 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:50.613042 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:50.614660 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:50.615166 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:50.615170 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:50.615173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:50.615263 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:50.615562 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:50.615565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:50.615567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:50.615775 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:50.615925 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:50.616182 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 17:13:50.616320 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 17:13:50.617901 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 21852
I20260811 17:13:50.631177 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 22004
I20260811 17:13:50.640843 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 22163
I20260811 17:13:50.660997 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 21742
2026-08-11T17:13:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2565 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:13:50 dist-test-slave-fd7l krb5kdc[22328](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:13:50 dist-test-slave-fd7l krb5kdc[22328](info): set up 2 sockets
Aug 11 17:13:50 dist-test-slave-fd7l krb5kdc[22328](info): commencing operation
krb5kdc: starting...
W20260811 17:13:51.800853 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:13:51 dist-test-slave-fd7l krb5kdc[22328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:13:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:13:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:13:51.945510 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:44659
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:41927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:44659
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:13:52.067677 22342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:13:52.068048 22342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:13:52.068132 22342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:13:52.072288 22342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:13:52.072404 22342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:13:52.072439 22342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:13:52.072482 22342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:13:52.072547 22342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:13:52.078087 22342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:41927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:44659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:44659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:13:52.079305 22342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:13:52.080432 22342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:13:52.090122 22348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:52.090116 22349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:52.090116 22351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:52.090211 22342 server_base.cc:1034] running on GCE node
I20260811 17:13:52.091221 22342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:13:52.092612 22342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:13:52.093834 22342 hybrid_clock.cc:648] HybridClock initialized: now 1786468432093813 us; error 38 us; skew 500 ppm
Aug 11 17:13:52 dist-test-slave-fd7l krb5kdc[22328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468432, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:13:52.102643 22342 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:13:52.103876 22342 webserver.cc:460] Webserver started at http://127.21.54.190:44945/ using document root <none> and password file <none>
I20260811 17:13:52.104599 22342 fs_manager.cc:359] Metadata directory not provided
I20260811 17:13:52.104678 22342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:13:52.104943 22342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:13:52.107311 22342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "49274efd6ed7486a994b3fee6587bcaf"
format_stamp: "Formatted at 2026-08-11 17:13:52 on dist-test-slave-fd7l"
server_key: "8ebd55841a0511ea45f72d792b74a3e3"
server_key_iv: "961adba38f2fcf49a174be3e29559a49"
server_key_version: "encryptionkey@0"
I20260811 17:13:52.107977 22342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "49274efd6ed7486a994b3fee6587bcaf"
format_stamp: "Formatted at 2026-08-11 17:13:52 on dist-test-slave-fd7l"
server_key: "8ebd55841a0511ea45f72d792b74a3e3"
server_key_iv: "961adba38f2fcf49a174be3e29559a49"
server_key_version: "encryptionkey@0"
I20260811 17:13:52.112212 22342 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 17:13:52.115096 22359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:52.116470 22342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 17:13:52.116691 22342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "49274efd6ed7486a994b3fee6587bcaf"
format_stamp: "Formatted at 2026-08-11 17:13:52 on dist-test-slave-fd7l"
server_key: "8ebd55841a0511ea45f72d792b74a3e3"
server_key_iv: "961adba38f2fcf49a174be3e29559a49"
server_key_version: "encryptionkey@0"
I20260811 17:13:52.116829 22342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:13:52.142156 22342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:13:52.145920 22342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:13:52.146169 22342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:13:52.154385 22342 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:44659
I20260811 17:13:52.154408 22420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:44659 every 8 connection(s)
I20260811 17:13:52.155514 22342 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:13:52.158759 22421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:52.164952 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 22342
I20260811 17:13:52.165130 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:13:52.165342 22421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf: Bootstrap starting.
I20260811 17:13:52.165447 21722 external_mini_cluster.cc:1428] Setting key a4977fae302f3bc06fdd0753015e89c9
I20260811 17:13:52.168336 22421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:52.169287 22421 log.cc:824] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf: Log is configured to *not* fsync() on all Append() calls
I20260811 17:13:52.171602 22421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf: No bootstrap required, opened a new log
Aug 11 17:13:52 dist-test-slave-fd7l krb5kdc[22328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:13:52.176566 22421 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49274efd6ed7486a994b3fee6587bcaf" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44659 } }
I20260811 17:13:52.176921 22421 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:52.176993 22421 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49274efd6ed7486a994b3fee6587bcaf, State: Initialized, Role: FOLLOWER
I20260811 17:13:52.177574 22421 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49274efd6ed7486a994b3fee6587bcaf" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44659 } }
I20260811 17:13:52.177736 22421 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:13:52.177826 22421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:13:52.177966 22421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:52.179042 22421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49274efd6ed7486a994b3fee6587bcaf" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44659 } }
I20260811 17:13:52.179438 22421 leader_election.cc:302] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49274efd6ed7486a994b3fee6587bcaf; no voters:
I20260811 17:13:52.179848 22421 leader_election.cc:288] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:13:52.179940 22426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:13:52.180164 22426 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf [term 1 LEADER]: Becoming Leader. State: Replica: 49274efd6ed7486a994b3fee6587bcaf, State: Running, Role: LEADER
I20260811 17:13:52.180495 22426 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49274efd6ed7486a994b3fee6587bcaf" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44659 } }
I20260811 17:13:52.181386 22421 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:13:52.182359 22428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49274efd6ed7486a994b3fee6587bcaf. Latest consensus state: current_term: 1 leader_uuid: "49274efd6ed7486a994b3fee6587bcaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49274efd6ed7486a994b3fee6587bcaf" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44659 } } }
I20260811 17:13:52.182487 22427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49274efd6ed7486a994b3fee6587bcaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49274efd6ed7486a994b3fee6587bcaf" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44659 } } }
I20260811 17:13:52.182488 22428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf [sys.catalog]: This master's current role is: LEADER
I20260811 17:13:52.182612 22427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf [sys.catalog]: This master's current role is: LEADER
I20260811 17:13:52.182829 22424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:52.166938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34816 (local address 127.21.54.190:44659)
0811 17:13:52.167487 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:52.167504 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:52.167588 (+ 84us) server_negotiation.cc:407] Connection header received
0811 17:13:52.168314 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:52.168335 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:52.168816 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:52.169203 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:52.170081 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:52.171106 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:52.172094 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:52.172670 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:52.172872 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:52.176019 (+ 3147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:52.176025 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:52.176044 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:52.176079 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:52.178442 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:52.179009 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:52.179017 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:52.179024 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:52.179205 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:52.179544 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:52.179547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:52.179549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:52.180162 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:52.181080 (+ 918us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:52.181215 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 17:13:52.181605 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":183,"threads_started":1}
W20260811 17:13:52.186846 22442 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:13:52.186954 22442 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:13:52.187021 22436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:13:52.187750 22436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:13:52.194272 22436 catalog_manager.cc:1347] Generated new cluster ID: 16be2025f9ca43bb82fd4f0ddb251c08
I20260811 17:13:52.194372 22436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:13:52.212227 22436 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:13:52.213353 22436 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:13:52.219266 22436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49274efd6ed7486a994b3fee6587bcaf: Generated new TSK 0
I20260811 17:13:52.220188 22436 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
I20260811 17:13:52.261487 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44659
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:41927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:13:52.385248 22449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:13:52.385558 22449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:13:52.385644 22449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:13:52.390004 22449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:13:52.390105 22449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:13:52.390202 22449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:13:52.395833 22449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:41927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44659
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:13:52.397076 22449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:13:52.398180 22449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:13:52.408890 22458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:52.408874 22460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:52.408850 22457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:52.409142 22449 server_base.cc:1034] running on GCE node
I20260811 17:13:52.409593 22449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:13:52.410290 22449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:13:52.411495 22449 hybrid_clock.cc:648] HybridClock initialized: now 1786468432411470 us; error 46 us; skew 500 ppm
Aug 11 17:13:52 dist-test-slave-fd7l krb5kdc[22328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468432, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:13:52.414700 22449 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:13:52.415697 22449 webserver.cc:460] Webserver started at http://127.21.54.129:44571/ using document root <none> and password file <none>
I20260811 17:13:52.416316 22449 fs_manager.cc:359] Metadata directory not provided
I20260811 17:13:52.416416 22449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:13:52.416713 22449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:13:52.418913 22449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "f1832528ada0401588d06609ac5b7c53"
format_stamp: "Formatted at 2026-08-11 17:13:52 on dist-test-slave-fd7l"
server_key: "90264120a6576d1de5d31a88efe37c76"
server_key_iv: "b726be67872840ecd50b15b035de7554"
server_key_version: "encryptionkey@0"
I20260811 17:13:52.419534 22449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "f1832528ada0401588d06609ac5b7c53"
format_stamp: "Formatted at 2026-08-11 17:13:52 on dist-test-slave-fd7l"
server_key: "90264120a6576d1de5d31a88efe37c76"
server_key_iv: "b726be67872840ecd50b15b035de7554"
server_key_version: "encryptionkey@0"
I20260811 17:13:52.423820 22449 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 17:13:52.426679 22471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:52.428025 22449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 17:13:52.428195 22449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "f1832528ada0401588d06609ac5b7c53"
format_stamp: "Formatted at 2026-08-11 17:13:52 on dist-test-slave-fd7l"
server_key: "90264120a6576d1de5d31a88efe37c76"
server_key_iv: "b726be67872840ecd50b15b035de7554"
server_key_version: "encryptionkey@0"
I20260811 17:13:52.428330 22449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:13:52.447969 22449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:13:52.451838 22449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:13:52.452090 22449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:13:52.452847 22449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:13:52.453745 22449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:13:52.453819 22449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:52.453891 22449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:13:52.453941 22449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:52.463852 22449 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:39927
I20260811 17:13:52.463877 22599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:39927 every 8 connection(s)
I20260811 17:13:52.464920 22449 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:13:52.470659 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 22449
I20260811 17:13:52.470765 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:13:52.471050 21722 external_mini_cluster.cc:1428] Setting key ba0c6b0a8c7d4737cff930a2c5c9565c
Aug 11 17:13:52 dist-test-slave-fd7l krb5kdc[22328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468432, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:13:52.481608 22424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:52.467036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:55101 (local address 127.21.54.190:44659)
0811 17:13:52.467206 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:52.467210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:52.468018 (+ 808us) server_negotiation.cc:407] Connection header received
0811 17:13:52.469045 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:52.469050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:52.469117 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:52.469232 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:52.470605 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:52.471327 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:52.472921 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:52.473180 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:52.473277 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:52.476428 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:52.476436 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:52.476442 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:52.476484 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:52.478672 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:52.479403 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:52.479409 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:52.479414 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:52.479527 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:52.479933 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:52.479939 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:52.479942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:52.480263 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:52.480393 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:52.481207 (+ 814us) server_negotiation.cc:299] Negotiation successful
0811 17:13:52.481372 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 17:13:52.482767 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:52.467356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:44659 (local address 127.21.54.129:55101)
0811 17:13:52.467851 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:52.467895 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:52.468819 (+ 924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:52.469424 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:52.469432 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:52.469828 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:52.470365 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:52.470387 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:52.471480 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:52.471483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:52.472761 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:52.472775 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:52.473366 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:52.473373 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:52.473623 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:52.474384 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:52.474413 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:52.476204 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:52.478883 (+ 2679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:52.478893 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:52.478920 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:52.479272 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:52.479647 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:52.479649 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:52.479651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:52.479822 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:52.480471 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:52.480477 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:52.480858 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 17:13:52.481122 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 17:13:52.481464 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":127,"threads_started":1}
WARNING: no policy specified for kudu/127.21.54.130@KRBTEST.COM; defaulting to no policy
I20260811 17:13:52.484628 22600 heartbeater.cc:347] Connected to a master server at 127.21.54.190:44659
I20260811 17:13:52.484934 22600 heartbeater.cc:464] Registering TS with master...
I20260811 17:13:52.485584 22600 heartbeater.cc:511] Master 127.21.54.190:44659 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.130@KRBTEST.COM" created.
I20260811 17:13:52.487540 22381 ts_manager.cc:194] Registered new tserver with Master: f1832528ada0401588d06609ac5b7c53 (127.21.54.129:39927)
I20260811 17:13:52.488886 22381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:55101
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
I20260811 17:13:52.507912 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44659
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:41927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:13:52.631821 22606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:13:52.632138 22606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:13:52.632187 22606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:13:52.637272 22606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:13:52.637380 22606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:13:52.637492 22606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:13:52.643206 22606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:41927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44659
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:13:52.644464 22606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:13:52.645737 22606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:13:52.656456 22612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:52.656625 22611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:52.656461 22615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:52.656754 22606 server_base.cc:1034] running on GCE node
I20260811 17:13:52.657284 22606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:13:52.657963 22606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:13:52.659174 22606 hybrid_clock.cc:648] HybridClock initialized: now 1786468432659153 us; error 33 us; skew 500 ppm
Aug 11 17:13:52 dist-test-slave-fd7l krb5kdc[22328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468432, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:13:52.662691 22606 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:13:52.663735 22606 webserver.cc:460] Webserver started at http://127.21.54.130:33107/ using document root <none> and password file <none>
I20260811 17:13:52.664413 22606 fs_manager.cc:359] Metadata directory not provided
I20260811 17:13:52.664485 22606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:13:52.664780 22606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:13:52.667079 22606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "da6e4a73dea4478e9a78c0bbfb6c4af8"
format_stamp: "Formatted at 2026-08-11 17:13:52 on dist-test-slave-fd7l"
server_key: "d20f767256a30362f71f264d7587ed1c"
server_key_iv: "c9b67ea5336821307a54e5292ac298f0"
server_key_version: "encryptionkey@0"
I20260811 17:13:52.667726 22606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "da6e4a73dea4478e9a78c0bbfb6c4af8"
format_stamp: "Formatted at 2026-08-11 17:13:52 on dist-test-slave-fd7l"
server_key: "d20f767256a30362f71f264d7587ed1c"
server_key_iv: "c9b67ea5336821307a54e5292ac298f0"
server_key_version: "encryptionkey@0"
I20260811 17:13:52.672255 22606 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:13:52.675186 22624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:52.677047 22606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:13:52.677212 22606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "da6e4a73dea4478e9a78c0bbfb6c4af8"
format_stamp: "Formatted at 2026-08-11 17:13:52 on dist-test-slave-fd7l"
server_key: "d20f767256a30362f71f264d7587ed1c"
server_key_iv: "c9b67ea5336821307a54e5292ac298f0"
server_key_version: "encryptionkey@0"
I20260811 17:13:52.677351 22606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:13:52.694329 22606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:13:52.698217 22606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:13:52.698451 22606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:13:52.699150 22606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:13:52.700075 22606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:13:52.700135 22606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:52.700202 22606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:13:52.700235 22606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:52.710021 22606 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:37199
I20260811 17:13:52.710098 22749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:37199 every 8 connection(s)
I20260811 17:13:52.711117 22606 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:13:52.716874 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 22606
I20260811 17:13:52.716970 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:13:52.717267 21722 external_mini_cluster.cc:1428] Setting key f8255c587c892948dd350c675fadc736
Aug 11 17:13:52 dist-test-slave-fd7l krb5kdc[22328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468432, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:13:52.727885 22424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:52.713269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:47551 (local address 127.21.54.190:44659)
0811 17:13:52.713422 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:52.713427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:52.714365 (+ 938us) server_negotiation.cc:407] Connection header received
0811 17:13:52.715374 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:52.715378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:52.715450 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:52.715627 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:52.717121 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:52.717771 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:52.719438 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:52.719700 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:52.719828 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:52.723099 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:52.723107 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:52.723114 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:52.723152 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:52.725267 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:52.725881 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:52.725889 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:52.725894 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:52.726011 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:52.726393 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:52.726400 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:52.726404 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:52.726741 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:52.726876 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:52.727487 (+ 611us) server_negotiation.cc:299] Negotiation successful
0811 17:13:52.727668 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 17:13:52.728868 22753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:52.713659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:44659 (local address 127.21.54.130:47551)
0811 17:13:52.714190 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:52.714240 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:52.715131 (+ 891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:52.715778 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:52.715791 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:52.716246 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:52.716865 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:52.716881 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:52.717943 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:52.717947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:52.719299 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:52.719310 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:52.719875 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:52.719880 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:52.720103 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:52.720940 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:52.720992 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:52.722908 (+ 1916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:52.725413 (+ 2505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:52.725419 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:52.725434 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:52.725759 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:52.726130 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:52.726132 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:52.726134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:52.726283 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:52.726897 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:52.726903 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:52.727191 (+ 288us) client_negotiation.cc:769] Sending connection context
0811 17:13:52.727385 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 17:13:52.727749 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":120,"threads_started":1}
WARNING: no policy specified for kudu/127.21.54.131@KRBTEST.COM; defaulting to no policy
I20260811 17:13:52.730235 22751 heartbeater.cc:347] Connected to a master server at 127.21.54.190:44659
I20260811 17:13:52.730543 22751 heartbeater.cc:464] Registering TS with master...
I20260811 17:13:52.731102 22751 heartbeater.cc:511] Master 127.21.54.190:44659 requested a full tablet report, sending...
I20260811 17:13:52.732350 22381 ts_manager.cc:194] Registered new tserver with Master: da6e4a73dea4478e9a78c0bbfb6c4af8 (127.21.54.130:37199)
I20260811 17:13:52.733157 22381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:47551
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
I20260811 17:13:52.752719 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44659
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:41927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:13:52.875541 22757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:13:52.875862 22757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:13:52.875911 22757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:13:52.880228 22757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:13:52.880319 22757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:13:52.880383 22757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:13:52.886933 22757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:41927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44659
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:13:52.888197 22757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:13:52.889401 22757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:13:52.900228 22764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:52.900254 22766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:52.900360 22757 server_base.cc:1034] running on GCE node
W20260811 17:13:52.900228 22763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:52.901156 22757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:13:52.901973 22757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:13:52.903189 22757 hybrid_clock.cc:648] HybridClock initialized: now 1786468432903166 us; error 40 us; skew 500 ppm
Aug 11 17:13:52 dist-test-slave-fd7l krb5kdc[22328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468432, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:13:52.906526 22757 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:13:52.907653 22757 webserver.cc:460] Webserver started at http://127.21.54.131:43105/ using document root <none> and password file <none>
I20260811 17:13:52.908306 22757 fs_manager.cc:359] Metadata directory not provided
I20260811 17:13:52.908407 22757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:13:52.908746 22757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:13:52.911103 22757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "9e92ea7a79e8432ab8dee8473e2ad6dd"
format_stamp: "Formatted at 2026-08-11 17:13:52 on dist-test-slave-fd7l"
server_key: "2a46d9d7ab9b81b413bfa606f9605376"
server_key_iv: "464ecc0f4a873ec158f5661178eae4aa"
server_key_version: "encryptionkey@0"
I20260811 17:13:52.912038 22757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "9e92ea7a79e8432ab8dee8473e2ad6dd"
format_stamp: "Formatted at 2026-08-11 17:13:52 on dist-test-slave-fd7l"
server_key: "2a46d9d7ab9b81b413bfa606f9605376"
server_key_iv: "464ecc0f4a873ec158f5661178eae4aa"
server_key_version: "encryptionkey@0"
I20260811 17:13:52.916666 22757 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260811 17:13:52.919685 22775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:52.921315 22757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:13:52.921478 22757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "9e92ea7a79e8432ab8dee8473e2ad6dd"
format_stamp: "Formatted at 2026-08-11 17:13:52 on dist-test-slave-fd7l"
server_key: "2a46d9d7ab9b81b413bfa606f9605376"
server_key_iv: "464ecc0f4a873ec158f5661178eae4aa"
server_key_version: "encryptionkey@0"
I20260811 17:13:52.921697 22757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:13:52.943374 22757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:13:52.947085 22757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:13:52.947342 22757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:13:52.948086 22757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:13:52.949110 22757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:13:52.949190 22757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:52.949275 22757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:13:52.949323 22757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:52.959910 22757 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:35351
I20260811 17:13:52.959932 22899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:35351 every 8 connection(s)
I20260811 17:13:52.961017 22757 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:13:52.962741 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 22757
I20260811 17:13:52.962898 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:13:52.963223 21722 external_mini_cluster.cc:1428] Setting key 006cf3fd81b1ab9e39958c2cd34a795c
Aug 11 17:13:52 dist-test-slave-fd7l krb5kdc[22328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468432, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:13:52.976397 22424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:52.963404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:58179 (local address 127.21.54.190:44659)
0811 17:13:52.963583 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:52.963588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:52.964660 (+ 1072us) server_negotiation.cc:407] Connection header received
0811 17:13:52.965758 (+ 1098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:52.965761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:52.965821 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:52.965903 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:52.967203 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:52.967796 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:52.969229 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:52.969490 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:52.969620 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:52.972432 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:52.972436 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:52.972438 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:52.972466 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:52.974041 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:52.974633 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:52.974639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:52.974644 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:52.974738 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:52.975159 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:52.975163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:52.975165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:52.975414 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:52.975534 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:52.976081 (+ 547us) server_negotiation.cc:299] Negotiation successful
0811 17:13:52.976225 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 17:13:52.977496 22902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:52.963881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:44659 (local address 127.21.54.131:58179)
0811 17:13:52.964453 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:52.964545 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:52.965510 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:52.966096 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:52.966105 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:52.966509 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:52.967045 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:52.967060 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:52.967996 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:52.968000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:52.969085 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:52.969098 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:52.969673 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:52.969678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:52.969872 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:52.970497 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:52.970516 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:52.972225 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:52.974191 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:52.974199 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:52.974217 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:52.974516 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:52.974865 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:52.974871 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:52.974873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:52.975045 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:52.975532 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:52.975541 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:52.975825 (+ 284us) client_negotiation.cc:769] Sending connection context
0811 17:13:52.976039 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 17:13:52.976414 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":132,"threads_started":1}
I20260811 17:13:52.978822 22900 heartbeater.cc:347] Connected to a master server at 127.21.54.190:44659
I20260811 17:13:52.979148 22900 heartbeater.cc:464] Registering TS with master...
I20260811 17:13:52.979740 22900 heartbeater.cc:511] Master 127.21.54.190:44659 requested a full tablet report, sending...
I20260811 17:13:52.981041 22381 ts_manager.cc:194] Registered new tserver with Master: 9e92ea7a79e8432ab8dee8473e2ad6dd (127.21.54.131:35351)
I20260811 17:13:52.981843 22381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:58179
I20260811 17:13:52.990803 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:13:53.006870 22424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:52.997716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34826 (local address 127.21.54.190:44659)
0811 17:13:52.997917 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:52.997921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:52.998038 (+ 117us) server_negotiation.cc:407] Connection header received
0811 17:13:52.998204 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:52.998208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:52.998264 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:52.998355 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:52.999214 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:52.999844 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.000773 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.001006 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.001098 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:53.002580 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:53.002584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:53.002586 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:53.002619 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:53.004598 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:53.005028 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:53.005032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:53.005033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:53.005138 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:53.005811 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:53.005814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:53.005815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:53.006109 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:53.006276 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:53.006596 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 17:13:53.006718 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 17:13:53.013993 22381 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:13:53.017961 22381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:13:53.037390 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:53.030423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:39927 (local address 127.0.0.1:35600)
0811 17:13:53.030929 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:53.030943 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:53.031037 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:53.032151 (+ 1114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:53.032155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:53.032173 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:53.032300 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:53.032309 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.034028 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.034031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:53.035603 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:53.035613 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.036697 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.036701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:53.036872 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:53.036923 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 17:13:53.037054 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 17:13:53.037196 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":64,"thread_start_us":119,"threads_started":1}
I20260811 17:13:53.037703 22912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:53.030226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:37199 (local address 127.0.0.1:43496)
0811 17:13:53.030663 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:53.030725 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:53.030901 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:53.031828 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:53.031835 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:53.031863 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:53.032071 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:53.032083 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.033833 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.033836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:53.035781 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:53.035788 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.037062 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.037065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:53.037360 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:53.037385 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 17:13:53.037472 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 17:13:53.037561 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":350,"thread_start_us":126,"threads_started":1}
I20260811 17:13:53.038093 22915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:53.031152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35600 (local address 127.21.54.129:39927)
0811 17:13:53.031558 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:53.031568 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:53.031599 (+ 31us) server_negotiation.cc:407] Connection header received
0811 17:13:53.031690 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:53.031701 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:53.031926 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:53.032566 (+ 640us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:53.032712 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.033886 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.035758 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.036494 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.036684 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:53.036951 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:53.037820 (+ 869us) server_negotiation.cc:299] Negotiation successful
0811 17:13:53.037947 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":78,"threads_started":1}
I20260811 17:13:53.038157 22914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:53.030387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43496 (local address 127.21.54.130:37199)
0811 17:13:53.031261 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:53.031275 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:53.031301 (+ 26us) server_negotiation.cc:407] Connection header received
0811 17:13:53.031385 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:53.031393 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:53.031639 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:53.031824 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:53.032230 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.033651 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.035920 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.036888 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.037008 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:53.037193 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:53.037897 (+ 704us) server_negotiation.cc:299] Negotiation successful
0811 17:13:53.038009 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":766,"thread_start_us":150,"threads_started":1}
I20260811 17:13:53.039004 22916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:53.031744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:35351 (local address 127.0.0.1:60036)
0811 17:13:53.032174 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:53.032187 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:53.032265 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:53.034635 (+ 2370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:53.034638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:53.034661 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:53.034747 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:53.034753 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.035957 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.035969 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:53.037294 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:53.037306 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.038551 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.038555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:53.038649 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:53.038675 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 17:13:53.038778 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 17:13:53.038846 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":93,"threads_started":1}
I20260811 17:13:53.040692 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:53.032189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60036 (local address 127.21.54.131:35351)
0811 17:13:53.034050 (+ 1861us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:53.034065 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:53.034093 (+ 28us) server_negotiation.cc:407] Connection header received
0811 17:13:53.034178 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:53.034184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:53.034386 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:53.034550 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:53.034916 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.035836 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.037455 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.038397 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.038531 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:53.038695 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:53.039002 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 17:13:53.039111 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1724,"thread_start_us":110,"threads_started":1}
I20260811 17:13:53.041587 22524 tablet_service.cc:1467] Processing CreateTablet for tablet 5bda2443f70245c297917d2c94db5f0e (DEFAULT_TABLE table=test-table [id=898ef34b7c774f61a083f8d3733f3131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:13:53.041569 22677 tablet_service.cc:1467] Processing CreateTablet for tablet 5bda2443f70245c297917d2c94db5f0e (DEFAULT_TABLE table=test-table [id=898ef34b7c774f61a083f8d3733f3131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:13:53.042999 22524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bda2443f70245c297917d2c94db5f0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:53.043013 22677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bda2443f70245c297917d2c94db5f0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:53.043955 22826 tablet_service.cc:1467] Processing CreateTablet for tablet 5bda2443f70245c297917d2c94db5f0e (DEFAULT_TABLE table=test-table [id=898ef34b7c774f61a083f8d3733f3131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:13:53.045269 22826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bda2443f70245c297917d2c94db5f0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:53.050405 22918 tablet_bootstrap.cc:492] T 5bda2443f70245c297917d2c94db5f0e P da6e4a73dea4478e9a78c0bbfb6c4af8: Bootstrap starting.
I20260811 17:13:53.051949 22919 tablet_bootstrap.cc:492] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53: Bootstrap starting.
I20260811 17:13:53.053279 22921 tablet_bootstrap.cc:492] T 5bda2443f70245c297917d2c94db5f0e P 9e92ea7a79e8432ab8dee8473e2ad6dd: Bootstrap starting.
I20260811 17:13:53.054600 22919 tablet_bootstrap.cc:654] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:53.054616 22918 tablet_bootstrap.cc:654] T 5bda2443f70245c297917d2c94db5f0e P da6e4a73dea4478e9a78c0bbfb6c4af8: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:53.055497 22921 tablet_bootstrap.cc:654] T 5bda2443f70245c297917d2c94db5f0e P 9e92ea7a79e8432ab8dee8473e2ad6dd: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:53.055655 22918 log.cc:824] T 5bda2443f70245c297917d2c94db5f0e P da6e4a73dea4478e9a78c0bbfb6c4af8: Log is configured to *not* fsync() on all Append() calls
I20260811 17:13:53.055655 22919 log.cc:824] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53: Log is configured to *not* fsync() on all Append() calls
I20260811 17:13:53.057070 22921 log.cc:824] T 5bda2443f70245c297917d2c94db5f0e P 9e92ea7a79e8432ab8dee8473e2ad6dd: Log is configured to *not* fsync() on all Append() calls
I20260811 17:13:53.058192 22918 tablet_bootstrap.cc:492] T 5bda2443f70245c297917d2c94db5f0e P da6e4a73dea4478e9a78c0bbfb6c4af8: No bootstrap required, opened a new log
I20260811 17:13:53.058192 22919 tablet_bootstrap.cc:492] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53: No bootstrap required, opened a new log
I20260811 17:13:53.058462 22918 ts_tablet_manager.cc:1397] T 5bda2443f70245c297917d2c94db5f0e P da6e4a73dea4478e9a78c0bbfb6c4af8: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 17:13:53.058462 22919 ts_tablet_manager.cc:1397] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 17:13:53.059043 22921 tablet_bootstrap.cc:492] T 5bda2443f70245c297917d2c94db5f0e P 9e92ea7a79e8432ab8dee8473e2ad6dd: No bootstrap required, opened a new log
I20260811 17:13:53.059263 22921 ts_tablet_manager.cc:1397] T 5bda2443f70245c297917d2c94db5f0e P 9e92ea7a79e8432ab8dee8473e2ad6dd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 17:13:53.063237 22921 raft_consensus.cc:348] T 5bda2443f70245c297917d2c94db5f0e P 9e92ea7a79e8432ab8dee8473e2ad6dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da6e4a73dea4478e9a78c0bbfb6c4af8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37199 } } peers { permanent_uuid: "f1832528ada0401588d06609ac5b7c53" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39927 } } peers { permanent_uuid: "9e92ea7a79e8432ab8dee8473e2ad6dd" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35351 } }
I20260811 17:13:53.063549 22921 raft_consensus.cc:374] T 5bda2443f70245c297917d2c94db5f0e P 9e92ea7a79e8432ab8dee8473e2ad6dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:53.063618 22921 raft_consensus.cc:729] T 5bda2443f70245c297917d2c94db5f0e P 9e92ea7a79e8432ab8dee8473e2ad6dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e92ea7a79e8432ab8dee8473e2ad6dd, State: Initialized, Role: FOLLOWER
I20260811 17:13:53.063572 22918 raft_consensus.cc:348] T 5bda2443f70245c297917d2c94db5f0e P da6e4a73dea4478e9a78c0bbfb6c4af8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da6e4a73dea4478e9a78c0bbfb6c4af8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37199 } } peers { permanent_uuid: "f1832528ada0401588d06609ac5b7c53" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39927 } } peers { permanent_uuid: "9e92ea7a79e8432ab8dee8473e2ad6dd" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35351 } }
I20260811 17:13:53.063880 22918 raft_consensus.cc:374] T 5bda2443f70245c297917d2c94db5f0e P da6e4a73dea4478e9a78c0bbfb6c4af8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:53.063948 22918 raft_consensus.cc:729] T 5bda2443f70245c297917d2c94db5f0e P da6e4a73dea4478e9a78c0bbfb6c4af8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da6e4a73dea4478e9a78c0bbfb6c4af8, State: Initialized, Role: FOLLOWER
I20260811 17:13:53.064136 22921 consensus_queue.cc:260] T 5bda2443f70245c297917d2c94db5f0e P 9e92ea7a79e8432ab8dee8473e2ad6dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da6e4a73dea4478e9a78c0bbfb6c4af8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37199 } } peers { permanent_uuid: "f1832528ada0401588d06609ac5b7c53" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39927 } } peers { permanent_uuid: "9e92ea7a79e8432ab8dee8473e2ad6dd" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35351 } }
I20260811 17:13:53.064283 22919 raft_consensus.cc:348] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da6e4a73dea4478e9a78c0bbfb6c4af8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37199 } } peers { permanent_uuid: "f1832528ada0401588d06609ac5b7c53" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39927 } } peers { permanent_uuid: "9e92ea7a79e8432ab8dee8473e2ad6dd" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35351 } }
I20260811 17:13:53.064472 22918 consensus_queue.cc:260] T 5bda2443f70245c297917d2c94db5f0e P da6e4a73dea4478e9a78c0bbfb6c4af8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da6e4a73dea4478e9a78c0bbfb6c4af8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37199 } } peers { permanent_uuid: "f1832528ada0401588d06609ac5b7c53" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39927 } } peers { permanent_uuid: "9e92ea7a79e8432ab8dee8473e2ad6dd" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35351 } }
I20260811 17:13:53.064610 22919 raft_consensus.cc:374] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:53.064702 22919 raft_consensus.cc:729] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1832528ada0401588d06609ac5b7c53, State: Initialized, Role: FOLLOWER
I20260811 17:13:53.065280 22900 heartbeater.cc:503] Master 127.21.54.190:44659 was elected leader, sending a full tablet report...
I20260811 17:13:53.065572 22751 heartbeater.cc:503] Master 127.21.54.190:44659 was elected leader, sending a full tablet report...
I20260811 17:13:53.065644 22921 ts_tablet_manager.cc:1428] T 5bda2443f70245c297917d2c94db5f0e P 9e92ea7a79e8432ab8dee8473e2ad6dd: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 17:13:53.065708 22918 ts_tablet_manager.cc:1428] T 5bda2443f70245c297917d2c94db5f0e P da6e4a73dea4478e9a78c0bbfb6c4af8: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 17:13:53.066718 22919 consensus_queue.cc:260] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da6e4a73dea4478e9a78c0bbfb6c4af8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37199 } } peers { permanent_uuid: "f1832528ada0401588d06609ac5b7c53" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39927 } } peers { permanent_uuid: "9e92ea7a79e8432ab8dee8473e2ad6dd" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35351 } }
I20260811 17:13:53.067394 22600 heartbeater.cc:503] Master 127.21.54.190:44659 was elected leader, sending a full tablet report...
I20260811 17:13:53.067955 22919 ts_tablet_manager.cc:1428] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53: Time spent starting tablet: real 0.009s user 0.004s sys 0.004s
W20260811 17:13:53.212342 22901 tablet.cc:2369] T 5bda2443f70245c297917d2c94db5f0e P 9e92ea7a79e8432ab8dee8473e2ad6dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:13:53.212368 22752 tablet.cc:2369] T 5bda2443f70245c297917d2c94db5f0e P da6e4a73dea4478e9a78c0bbfb6c4af8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:13:53.216374 22601 tablet.cc:2369] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:13:53.322075 22928 raft_consensus.cc:482] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:13:53.322422 22928 raft_consensus.cc:504] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da6e4a73dea4478e9a78c0bbfb6c4af8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37199 } } peers { permanent_uuid: "f1832528ada0401588d06609ac5b7c53" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39927 } } peers { permanent_uuid: "9e92ea7a79e8432ab8dee8473e2ad6dd" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35351 } }
I20260811 17:13:53.323731 22928 leader_election.cc:288] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da6e4a73dea4478e9a78c0bbfb6c4af8 (127.21.54.130:37199), 9e92ea7a79e8432ab8dee8473e2ad6dd (127.21.54.131:35351)
I20260811 17:13:53.329133 22930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:53.324900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:35351 (local address 127.21.54.129:36257)
0811 17:13:53.325364 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:53.325380 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:53.325557 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:53.325882 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:53.325884 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:53.325899 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:53.326022 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:53.326029 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.327033 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.327038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:53.327864 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:53.327874 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.328641 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.328645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:53.328747 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:53.328778 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 17:13:53.328871 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 17:13:53.328952 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":3,"spinlock_wait_cycles":64512,"thread_start_us":138,"threads_started":1}
I20260811 17:13:53.329104 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:53.325212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:36257 (local address 127.21.54.131:35351)
0811 17:13:53.325365 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:53.325372 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:53.325487 (+ 115us) server_negotiation.cc:407] Connection header received
0811 17:13:53.325697 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:53.325700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:53.325770 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:53.325849 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:53.326163 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.326906 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.328009 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.328448 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.328645 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:53.328703 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:53.328901 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 17:13:53.328977 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 17:13:53.329205 22914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:53.324205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:38807 (local address 127.21.54.130:37199)
0811 17:13:53.324408 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:53.324412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:53.325000 (+ 588us) server_negotiation.cc:407] Connection header received
0811 17:13:53.325112 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:53.325115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:53.325175 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:53.325258 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:53.325865 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.326690 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.328114 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.328625 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.328764 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:53.328824 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:53.329013 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 17:13:53.329095 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 17:13:53.329445 22929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:53.324131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:37199 (local address 127.21.54.129:38807)
0811 17:13:53.324702 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:53.324724 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:53.324897 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:53.325468 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:53.325471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:53.325539 (+ 68us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:53.325717 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:53.325724 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.326905 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.326908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:53.327855 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:53.327866 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.328757 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.328760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:53.328874 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:53.328892 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 17:13:53.328980 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 17:13:53.329054 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":23,"spinlock_wait_cycles":4864,"thread_start_us":241,"threads_started":1}
I20260811 17:13:53.329943 22849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bda2443f70245c297917d2c94db5f0e" candidate_uuid: "f1832528ada0401588d06609ac5b7c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e92ea7a79e8432ab8dee8473e2ad6dd" is_pre_election: true
I20260811 17:13:53.330214 22697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bda2443f70245c297917d2c94db5f0e" candidate_uuid: "f1832528ada0401588d06609ac5b7c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da6e4a73dea4478e9a78c0bbfb6c4af8" is_pre_election: true
I20260811 17:13:53.330438 22849 raft_consensus.cc:2454] T 5bda2443f70245c297917d2c94db5f0e P 9e92ea7a79e8432ab8dee8473e2ad6dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1832528ada0401588d06609ac5b7c53 in term 0.
I20260811 17:13:53.330646 22697 raft_consensus.cc:2454] T 5bda2443f70245c297917d2c94db5f0e P da6e4a73dea4478e9a78c0bbfb6c4af8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1832528ada0401588d06609ac5b7c53 in term 0.
I20260811 17:13:53.331143 22472 leader_election.cc:302] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e92ea7a79e8432ab8dee8473e2ad6dd, f1832528ada0401588d06609ac5b7c53; no voters:
I20260811 17:13:53.331414 22928 raft_consensus.cc:2781] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:13:53.331526 22928 raft_consensus.cc:482] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:13:53.331565 22928 raft_consensus.cc:3037] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:53.333000 22928 raft_consensus.cc:504] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da6e4a73dea4478e9a78c0bbfb6c4af8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37199 } } peers { permanent_uuid: "f1832528ada0401588d06609ac5b7c53" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39927 } } peers { permanent_uuid: "9e92ea7a79e8432ab8dee8473e2ad6dd" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35351 } }
I20260811 17:13:53.333427 22928 leader_election.cc:288] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53 [CANDIDATE]: Term 1 election: Requested vote from peers da6e4a73dea4478e9a78c0bbfb6c4af8 (127.21.54.130:37199), 9e92ea7a79e8432ab8dee8473e2ad6dd (127.21.54.131:35351)
I20260811 17:13:53.334019 22697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bda2443f70245c297917d2c94db5f0e" candidate_uuid: "f1832528ada0401588d06609ac5b7c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da6e4a73dea4478e9a78c0bbfb6c4af8"
I20260811 17:13:53.334113 22849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bda2443f70245c297917d2c94db5f0e" candidate_uuid: "f1832528ada0401588d06609ac5b7c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e92ea7a79e8432ab8dee8473e2ad6dd"
I20260811 17:13:53.334162 22697 raft_consensus.cc:3037] T 5bda2443f70245c297917d2c94db5f0e P da6e4a73dea4478e9a78c0bbfb6c4af8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:53.334314 22849 raft_consensus.cc:3037] T 5bda2443f70245c297917d2c94db5f0e P 9e92ea7a79e8432ab8dee8473e2ad6dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:53.335479 22849 raft_consensus.cc:2454] T 5bda2443f70245c297917d2c94db5f0e P 9e92ea7a79e8432ab8dee8473e2ad6dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1832528ada0401588d06609ac5b7c53 in term 1.
I20260811 17:13:53.335480 22697 raft_consensus.cc:2454] T 5bda2443f70245c297917d2c94db5f0e P da6e4a73dea4478e9a78c0bbfb6c4af8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1832528ada0401588d06609ac5b7c53 in term 1.
I20260811 17:13:53.335918 22472 leader_election.cc:302] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e92ea7a79e8432ab8dee8473e2ad6dd, f1832528ada0401588d06609ac5b7c53; no voters:
I20260811 17:13:53.336230 22928 raft_consensus.cc:2781] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:13:53.336601 22928 raft_consensus.cc:686] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53 [term 1 LEADER]: Becoming Leader. State: Replica: f1832528ada0401588d06609ac5b7c53, State: Running, Role: LEADER
I20260811 17:13:53.337065 22928 consensus_queue.cc:237] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da6e4a73dea4478e9a78c0bbfb6c4af8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37199 } } peers { permanent_uuid: "f1832528ada0401588d06609ac5b7c53" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39927 } } peers { permanent_uuid: "9e92ea7a79e8432ab8dee8473e2ad6dd" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35351 } }
I20260811 17:13:53.341059 22381 catalog_manager.cc:5495] T 5bda2443f70245c297917d2c94db5f0e P f1832528ada0401588d06609ac5b7c53 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1832528ada0401588d06609ac5b7c53 (127.21.54.129). New cstate: current_term: 1 leader_uuid: "f1832528ada0401588d06609ac5b7c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da6e4a73dea4478e9a78c0bbfb6c4af8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1832528ada0401588d06609ac5b7c53" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e92ea7a79e8432ab8dee8473e2ad6dd" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35351 } health_report { overall_health: UNKNOWN } } }
Aug 11 17:13:53 dist-test-slave-fd7l krb5kdc[22328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.129@KRBTEST.COM
I20260811 17:13:53.409965 22915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:53.398771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43590 (local address 127.21.54.129:39927)
0811 17:13:53.399010 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:53.399018 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:53.399228 (+ 210us) server_negotiation.cc:407] Connection header received
0811 17:13:53.399454 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:53.399458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:53.399507 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:53.399734 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:53.400619 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.401206 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.402200 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.402448 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.402571 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:53.405740 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:53.405756 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:53.405773 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:53.405800 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:53.407916 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:53.408445 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:53.408451 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:53.408452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:53.408620 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:53.408920 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:53.408923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:53.408925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:53.409215 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:53.409457 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:53.409662 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 17:13:53.409796 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
Aug 11 17:13:53 dist-test-slave-fd7l krb5kdc[22328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468433, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 17:13:53 dist-test-slave-fd7l krb5kdc[22328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468433, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.54.129@KRBTEST.COM
I20260811 17:13:53.440260 22915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:53.431161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43604 (local address 127.21.54.129:39927)
0811 17:13:53.431373 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:53.431377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:53.431472 (+ 95us) server_negotiation.cc:407] Connection header received
0811 17:13:53.431575 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:53.431577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:53.431626 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:53.431704 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:53.432653 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.433236 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.434105 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.434335 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.434420 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:53.437092 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:53.437095 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:53.437097 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:53.437123 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:53.438452 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:53.438868 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:53.438870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:53.438872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:53.438956 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:53.439207 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:53.439209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:53.439210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:53.439393 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:53.439497 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:53.439822 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 17:13:53.439927 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
Aug 11 17:13:53 dist-test-slave-fd7l krb5kdc[22328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 17:13:53 dist-test-slave-fd7l krb5kdc[22328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.129@KRBTEST.COM
I20260811 17:13:53.471735 22915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:53.460241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43616 (local address 127.21.54.129:39927)
0811 17:13:53.460495 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:53.460501 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:53.460698 (+ 197us) server_negotiation.cc:407] Connection header received
0811 17:13:53.460909 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:53.460912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:53.460958 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:53.461051 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:53.461791 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.462399 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.463336 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:53.463736 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:53.463838 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:53.467899 (+ 4061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:53.467903 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:53.467915 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:53.467946 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:53.469938 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:53.470405 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:53.470409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:53.470410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:53.470504 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:53.470829 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:53.470832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:53.470834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:53.471026 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:53.471189 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:53.471443 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 17:13:53.471568 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
W20260811 17:13:53.473758 22501 tablet_service.cc:2908] Rejecting scan request for tablet 5bda2443f70245c297917d2c94db5f0e: Uninitialized: clean time has not yet been initialized
I20260811 17:13:53.476693 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 22449
I20260811 17:13:53.491878 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 22606
I20260811 17:13:53.502744 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 22757
I20260811 17:13:53.510749 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 22342
2026-08-11T17:13:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2847 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:13:53 dist-test-slave-fd7l krb5kdc[22947](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:13:53 dist-test-slave-fd7l krb5kdc[22947](info): set up 2 sockets
Aug 11 17:13:53 dist-test-slave-fd7l krb5kdc[22947](info): commencing operation
krb5kdc: starting...
W20260811 17:13:54.667081 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:13:54 dist-test-slave-fd7l krb5kdc[22947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:13:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:13:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:13:54.816821 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:32939
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:37305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:32939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:13:54.936342 22962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:13:54.936723 22962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:13:54.936813 22962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:13:54.940873 22962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:13:54.940980 22962 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 17:13:54.941015 22962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:13:54.941056 22962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:13:54.941114 22962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:13:54.941154 22962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:13:54.947077 22962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:37305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:32939
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:32939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:13:54.948308 22962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:13:54.949476 22962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:13:54.959132 22973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:54.959132 22970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:54.959396 22962 server_base.cc:1034] running on GCE node
W20260811 17:13:54.959322 22969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:54.960176 22962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:13:54.961572 22962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:13:54.962765 22962 hybrid_clock.cc:648] HybridClock initialized: now 1786468434962743 us; error 36 us; skew 500 ppm
Aug 11 17:13:54 dist-test-slave-fd7l krb5kdc[22947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468434, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:13:54.966434 22962 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:13:54.967478 22962 webserver.cc:460] Webserver started at http://127.21.54.190:43569/ using document root <none> and password file <none>
I20260811 17:13:54.968076 22962 fs_manager.cc:359] Metadata directory not provided
I20260811 17:13:54.968139 22962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:13:54.968321 22962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:13:54.970587 22962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "6bf63d35a6004a8491b53136f51f2524"
format_stamp: "Formatted at 2026-08-11 17:13:54 on dist-test-slave-fd7l"
server_key: "a932c5c42d103fa7a85be5a0134dbdf2"
server_key_iv: "0396a4ef14a7fa1a2260a7be28681bad"
server_key_version: "encryptionkey@0"
I20260811 17:13:54.971138 22962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "6bf63d35a6004a8491b53136f51f2524"
format_stamp: "Formatted at 2026-08-11 17:13:54 on dist-test-slave-fd7l"
server_key: "a932c5c42d103fa7a85be5a0134dbdf2"
server_key_iv: "0396a4ef14a7fa1a2260a7be28681bad"
server_key_version: "encryptionkey@0"
I20260811 17:13:54.975391 22962 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 17:13:54.978232 22981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:54.979849 22962 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:13:54.980036 22962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "6bf63d35a6004a8491b53136f51f2524"
format_stamp: "Formatted at 2026-08-11 17:13:54 on dist-test-slave-fd7l"
server_key: "a932c5c42d103fa7a85be5a0134dbdf2"
server_key_iv: "0396a4ef14a7fa1a2260a7be28681bad"
server_key_version: "encryptionkey@0"
I20260811 17:13:54.980187 22962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:13:54.990391 22962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:13:54.994213 22962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:13:54.994406 22962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:13:55.006227 22962 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:32939
I20260811 17:13:55.006234 23052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:32939 every 8 connection(s)
I20260811 17:13:55.007406 22962 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:13:55.010861 23053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:55.015408 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 22962
I20260811 17:13:55.015511 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:13:55.015786 21722 external_mini_cluster.cc:1428] Setting key 8318efee073a158d8271cf8a396797d8
I20260811 17:13:55.017812 23053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524: Bootstrap starting.
I20260811 17:13:55.021044 23053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:55.022066 23053 log.cc:824] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524: Log is configured to *not* fsync() on all Append() calls
I20260811 17:13:55.024838 23053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524: No bootstrap required, opened a new log
Aug 11 17:13:55 dist-test-slave-fd7l krb5kdc[22947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:13:55.029768 23053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bf63d35a6004a8491b53136f51f2524" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 32939 } }
I20260811 17:13:55.030126 23053 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:55.030197 23053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bf63d35a6004a8491b53136f51f2524, State: Initialized, Role: FOLLOWER
I20260811 17:13:55.030794 23053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bf63d35a6004a8491b53136f51f2524" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 32939 } }
I20260811 17:13:55.031000 23053 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:13:55.031097 23053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:13:55.031206 23053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:55.032598 23053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bf63d35a6004a8491b53136f51f2524" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 32939 } }
I20260811 17:13:55.032996 23053 leader_election.cc:302] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bf63d35a6004a8491b53136f51f2524; no voters:
I20260811 17:13:55.033371 23053 leader_election.cc:288] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:13:55.033512 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.017523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44648 (local address 127.21.54.190:32939)
0811 17:13:55.018030 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:55.018060 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:55.018111 (+ 51us) server_negotiation.cc:407] Connection header received
0811 17:13:55.019071 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:55.019101 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:55.019593 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:55.020012 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:55.020939 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.022667 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.023723 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.024547 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.024736 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.027363 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:55.027371 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:55.027390 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:55.027427 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:55.029771 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:55.030274 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:55.030282 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:55.030289 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:55.030445 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:55.030765 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:55.030768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:55.030770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:55.031428 (+ 658us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:55.031649 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:55.031984 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 17:13:55.032350 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":136,"threads_started":1}
I20260811 17:13:55.033674 23060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:13:55.033969 23060 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524 [term 1 LEADER]: Becoming Leader. State: Replica: 6bf63d35a6004a8491b53136f51f2524, State: Running, Role: LEADER
I20260811 17:13:55.034344 23060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bf63d35a6004a8491b53136f51f2524" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 32939 } }
I20260811 17:13:55.034783 23053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:13:55.035701 23062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bf63d35a6004a8491b53136f51f2524. Latest consensus state: current_term: 1 leader_uuid: "6bf63d35a6004a8491b53136f51f2524" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bf63d35a6004a8491b53136f51f2524" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 32939 } } }
I20260811 17:13:55.035849 23062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524 [sys.catalog]: This master's current role is: LEADER
I20260811 17:13:55.036152 23061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bf63d35a6004a8491b53136f51f2524" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bf63d35a6004a8491b53136f51f2524" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 32939 } } }
I20260811 17:13:55.036265 23061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524 [sys.catalog]: This master's current role is: LEADER
I20260811 17:13:55.036916 23070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:13:55.040437 23070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:13:55.046938 23070 catalog_manager.cc:1347] Generated new cluster ID: 5278c67761bd454ca22d4f5177dd919e
I20260811 17:13:55.047040 23070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:13:55.066540 23070 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:13:55.067777 23070 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:13:55.075409 23070 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524: Generated new TSK 0
I20260811 17:13:55.076372 23070 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
I20260811 17:13:55.118098 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:32939
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:37305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:13:55.245883 23084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:13:55.246251 23084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:13:55.246345 23084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:13:55.250662 23084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:13:55.250801 23084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:13:55.250890 23084 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 17:13:55.251000 23084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:13:55.256747 23084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:37305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.54.190:32939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:13:55.258083 23084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:13:55.259269 23084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:13:55.270680 23091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:55.270680 23090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:55.270843 23084 server_base.cc:1034] running on GCE node
W20260811 17:13:55.270721 23094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:55.271473 23084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:13:55.272226 23084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:13:55.273460 23084 hybrid_clock.cc:648] HybridClock initialized: now 1786468435273402 us; error 68 us; skew 500 ppm
Aug 11 17:13:55 dist-test-slave-fd7l krb5kdc[22947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468435, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:13:55.276835 23084 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:13:55.277985 23084 webserver.cc:460] Webserver started at http://127.21.54.129:35867/ using document root <none> and password file <none>
I20260811 17:13:55.278641 23084 fs_manager.cc:359] Metadata directory not provided
I20260811 17:13:55.278740 23084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:13:55.278990 23084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:13:55.281296 23084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "863e594fec3847d4b8315d66885846e3"
format_stamp: "Formatted at 2026-08-11 17:13:55 on dist-test-slave-fd7l"
server_key: "34f5be3d9c1b5c8a0bc1dc29f306e8c0"
server_key_iv: "08798aa657e600edec95016ebb837de0"
server_key_version: "encryptionkey@0"
I20260811 17:13:55.281900 23084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "863e594fec3847d4b8315d66885846e3"
format_stamp: "Formatted at 2026-08-11 17:13:55 on dist-test-slave-fd7l"
server_key: "34f5be3d9c1b5c8a0bc1dc29f306e8c0"
server_key_iv: "08798aa657e600edec95016ebb837de0"
server_key_version: "encryptionkey@0"
I20260811 17:13:55.286208 23084 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:13:55.289063 23102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:55.290503 23084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:13:55.290660 23084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "863e594fec3847d4b8315d66885846e3"
format_stamp: "Formatted at 2026-08-11 17:13:55 on dist-test-slave-fd7l"
server_key: "34f5be3d9c1b5c8a0bc1dc29f306e8c0"
server_key_iv: "08798aa657e600edec95016ebb837de0"
server_key_version: "encryptionkey@0"
I20260811 17:13:55.290791 23084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:13:55.306854 23084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:13:55.310861 23084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:13:55.311065 23084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:13:55.312845 23084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:13:55.312909 23084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:55.313046 23084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:13:55.313120 23084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 17:13:55 dist-test-slave-fd7l krb5kdc[22947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468435, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:13:55.325062 23084 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:43523
I20260811 17:13:55.325091 23236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:43523 every 8 connection(s)
I20260811 17:13:55.326393 23084 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:13:55.327859 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 23084
I20260811 17:13:55.327994 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:13:55.328298 21722 external_mini_cluster.cc:1428] Setting key 1edf9417b63176a021ebf603d92cc2ea
I20260811 17:13:55.328984 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.313808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:41689 (local address 127.21.54.190:32939)
0811 17:13:55.313974 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:55.313979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:55.315034 (+ 1055us) server_negotiation.cc:407] Connection header received
0811 17:13:55.316448 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:55.316454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:55.316545 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:55.316683 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:55.318233 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.318794 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.320097 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.320418 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.320576 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.324140 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:55.324144 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:55.324146 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:55.324173 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:55.326091 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:55.326743 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:55.326750 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:55.326753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:55.326868 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:55.327472 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:55.327477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:55.327478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:55.327718 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:55.327844 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:55.328652 (+ 808us) server_negotiation.cc:299] Negotiation successful
0811 17:13:55.328815 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 17:13:55.329910 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.314299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:32939 (local address 127.21.54.129:41689)
0811 17:13:55.314830 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:55.314896 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:55.316163 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:55.316869 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:55.316877 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:55.317432 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:55.318063 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:55.318080 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.318935 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.318939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:55.319941 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:55.319952 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.320665 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.320673 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:55.320929 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.321708 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:55.321729 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:55.323931 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:55.326234 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:55.326239 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:55.326253 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:55.326620 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:55.327037 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:55.327060 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:55.327062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:55.327294 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:55.327861 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:55.327870 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:55.328202 (+ 332us) client_negotiation.cc:769] Sending connection context
0811 17:13:55.328421 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 17:13:55.328838 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":124,"threads_started":1}
I20260811 17:13:55.331347 23237 heartbeater.cc:347] Connected to a master server at 127.21.54.190:32939
I20260811 17:13:55.331548 23237 heartbeater.cc:464] Registering TS with master...
I20260811 17:13:55.332232 23237 heartbeater.cc:511] Master 127.21.54.190:32939 requested a full tablet report, sending...
I20260811 17:13:55.334227 23001 ts_manager.cc:194] Registered new tserver with Master: 863e594fec3847d4b8315d66885846e3 (127.21.54.129:43523)
I20260811 17:13:55.336041 23001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:41689
WARNING: no policy specified for kudu/127.21.54.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.130@KRBTEST.COM" created.
I20260811 17:13:55.345050 23247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.333858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:32939 (local address 127.21.54.129:50233)
0811 17:13:55.334148 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:55.334164 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:55.334261 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:55.334553 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:55.334556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:55.334758 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:55.334866 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:55.334871 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.336037 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.336043 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:55.339313 (+ 3270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:55.339329 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.339838 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.339842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:55.339972 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.340706 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:55.340721 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:55.341252 (+ 531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:55.343165 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:55.343172 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:55.343174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:55.343424 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:55.343765 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:55.343770 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:55.343772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:55.343872 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:55.344459 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:55.344461 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:55.344646 (+ 185us) client_negotiation.cc:769] Sending connection context
0811 17:13:55.344747 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 17:13:55.344881 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":123,"threads_started":1}
I20260811 17:13:55.345306 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.334167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:50233 (local address 127.21.54.190:32939)
0811 17:13:55.334293 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:55.334297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:55.334314 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:13:55.334356 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:55.334359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:55.334400 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:55.334498 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:55.334987 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.335874 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.339468 (+ 3594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.339702 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.339792 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.341414 (+ 1622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:55.341417 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:55.341419 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:55.341489 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:55.343021 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:55.343548 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:55.343552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:55.343554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:55.343658 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:55.344059 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:55.344061 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:55.344063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:55.344322 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:55.344451 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:55.344851 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 17:13:55.345034 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":150,"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
I20260811 17:13:55.364684 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:32939
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:37305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:13:55.487406 23250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:13:55.487701 23250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:13:55.487779 23250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:13:55.492256 23250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:13:55.492331 23250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:13:55.492372 23250 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 17:13:55.492457 23250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:13:55.498366 23250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:37305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.54.190:32939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:13:55.499564 23250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:13:55.500705 23250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:13:55.511179 23250 server_base.cc:1034] running on GCE node
W20260811 17:13:55.511121 23257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:55.511121 23256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:55.511122 23259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:55.512140 23250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:13:55.512933 23250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:13:55.514132 23250 hybrid_clock.cc:648] HybridClock initialized: now 1786468435514105 us; error 39 us; skew 500 ppm
Aug 11 17:13:55 dist-test-slave-fd7l krb5kdc[22947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468435, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:13:55.518263 23250 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:13:55.519351 23250 webserver.cc:460] Webserver started at http://127.21.54.130:43587/ using document root <none> and password file <none>
I20260811 17:13:55.519992 23250 fs_manager.cc:359] Metadata directory not provided
I20260811 17:13:55.520097 23250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:13:55.520357 23250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:13:55.522701 23250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "e5f7a8e1701047fcbfa410b154313c72"
format_stamp: "Formatted at 2026-08-11 17:13:55 on dist-test-slave-fd7l"
server_key: "0c7577f3b0eb1fd46f591aa3cc0df1f5"
server_key_iv: "9618eecc734d2f22a3b5312ddedf34ff"
server_key_version: "encryptionkey@0"
I20260811 17:13:55.523451 23250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "e5f7a8e1701047fcbfa410b154313c72"
format_stamp: "Formatted at 2026-08-11 17:13:55 on dist-test-slave-fd7l"
server_key: "0c7577f3b0eb1fd46f591aa3cc0df1f5"
server_key_iv: "9618eecc734d2f22a3b5312ddedf34ff"
server_key_version: "encryptionkey@0"
I20260811 17:13:55.527657 23250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 17:13:55.530450 23268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:55.531883 23250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:13:55.532029 23250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "e5f7a8e1701047fcbfa410b154313c72"
format_stamp: "Formatted at 2026-08-11 17:13:55 on dist-test-slave-fd7l"
server_key: "0c7577f3b0eb1fd46f591aa3cc0df1f5"
server_key_iv: "9618eecc734d2f22a3b5312ddedf34ff"
server_key_version: "encryptionkey@0"
I20260811 17:13:55.532181 23250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:13:55.552080 23250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:13:55.555857 23250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:13:55.556084 23250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:13:55.557791 23250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:13:55.557871 23250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:55.557947 23250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:13:55.557991 23250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 17:13:55 dist-test-slave-fd7l krb5kdc[22947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468435, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:13:55.569573 23250 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:35013
I20260811 17:13:55.569952 23404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:35013 every 8 connection(s)
I20260811 17:13:55.570629 23250 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:13:55.573298 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 23250
I20260811 17:13:55.573407 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:13:55.573443 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.558700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:60449 (local address 127.21.54.190:32939)
0811 17:13:55.558871 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:55.558877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:55.559898 (+ 1021us) server_negotiation.cc:407] Connection header received
0811 17:13:55.561242 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:55.561246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:55.561323 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:55.561429 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:55.562948 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.563680 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.565688 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.565955 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.566112 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.569181 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:55.569186 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:55.569190 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:55.569250 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:55.570987 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:55.571690 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:55.571694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:55.571695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:55.571799 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:55.572229 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:55.572231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:55.572233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:55.572445 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:55.572597 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:55.573198 (+ 601us) server_negotiation.cc:299] Negotiation successful
0811 17:13:55.573318 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 17:13:55.573693 21722 external_mini_cluster.cc:1428] Setting key 265f5dd99ac135fe45733089e627dbdf
I20260811 17:13:55.574630 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.559181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:32939 (local address 127.21.54.130:60449)
0811 17:13:55.559697 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:55.559754 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:55.560956 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:55.561645 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:55.561655 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:55.562192 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:55.562774 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:55.562789 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.563826 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.563830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:55.565467 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:55.565505 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.566159 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.566163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:55.566395 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.567175 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:55.567205 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:55.568962 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:55.571124 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:55.571132 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:55.571149 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:55.571530 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:55.571901 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:55.571904 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:55.571905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:55.572133 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:55.572656 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:55.572661 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:55.572906 (+ 245us) client_negotiation.cc:769] Sending connection context
0811 17:13:55.573109 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 17:13:55.573504 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":153,"threads_started":1}
I20260811 17:13:55.576202 23405 heartbeater.cc:347] Connected to a master server at 127.21.54.190:32939
I20260811 17:13:55.576412 23405 heartbeater.cc:464] Registering TS with master...
I20260811 17:13:55.577111 23405 heartbeater.cc:511] Master 127.21.54.190:32939 requested a full tablet report, sending...
I20260811 17:13:55.578320 23002 ts_manager.cc:194] Registered new tserver with Master: e5f7a8e1701047fcbfa410b154313c72 (127.21.54.130:35013)
I20260811 17:13:55.579110 23002 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:60449
WARNING: no policy specified for kudu/127.21.54.131@KRBTEST.COM; defaulting to no policy
I20260811 17:13:55.588845 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.578895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:60329 (local address 127.21.54.190:32939)
0811 17:13:55.579001 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:55.579006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:55.579918 (+ 912us) server_negotiation.cc:407] Connection header received
0811 17:13:55.580031 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:55.580033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:55.580076 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:55.580151 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:55.580884 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.581741 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.583148 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.583372 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.583463 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.585025 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:55.585028 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:55.585030 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:55.585055 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:55.586900 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:55.587366 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:55.587369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:55.587370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:55.587452 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:55.587842 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:55.587850 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:55.587855 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:55.588138 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:55.588233 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:55.588569 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 17:13:55.588706 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 17:13:55.588876 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.578821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:32939 (local address 127.21.54.130:60329)
0811 17:13:55.579803 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:55.579818 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:55.579938 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:55.580196 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:55.580199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:55.580423 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:55.580637 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:55.580649 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.581913 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.581920 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:55.582992 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:55.583008 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.583505 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.583511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:55.583639 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.584313 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:55.584334 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:55.584862 (+ 528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:55.587022 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:55.587025 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:55.587027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:55.587246 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:55.587602 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:55.587604 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:55.587606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:55.587684 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:55.588244 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:55.588246 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:55.588336 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 17:13:55.588431 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 17:13:55.588691 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":3,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
I20260811 17:13:55.608055 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:32939
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:37305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:13:55.727247 23416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:13:55.727546 23416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:13:55.727629 23416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:13:55.731812 23416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:13:55.731886 23416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:13:55.731956 23416 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 17:13:55.732031 23416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:13:55.738076 23416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:37305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.54.190:32939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:13:55.739302 23416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:13:55.740391 23416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:13:55.750962 23416 server_base.cc:1034] running on GCE node
W20260811 17:13:55.750908 23421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:55.750979 23424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:55.750908 23422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:55.751705 23416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:13:55.752406 23416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:13:55.753639 23416 hybrid_clock.cc:648] HybridClock initialized: now 1786468435753622 us; error 31 us; skew 500 ppm
Aug 11 17:13:55 dist-test-slave-fd7l krb5kdc[22947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468435, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:13:55.756848 23416 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:13:55.757792 23416 webserver.cc:460] Webserver started at http://127.21.54.131:35691/ using document root <none> and password file <none>
I20260811 17:13:55.758351 23416 fs_manager.cc:359] Metadata directory not provided
I20260811 17:13:55.758414 23416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:13:55.758597 23416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:13:55.760854 23416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "d48ae49e8b43449a9a2cc635aab3687d"
format_stamp: "Formatted at 2026-08-11 17:13:55 on dist-test-slave-fd7l"
server_key: "a9e061689e6e2a59a0c8537054d64838"
server_key_iv: "dc84bb22c282033744de43f3547db487"
server_key_version: "encryptionkey@0"
I20260811 17:13:55.761468 23416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "d48ae49e8b43449a9a2cc635aab3687d"
format_stamp: "Formatted at 2026-08-11 17:13:55 on dist-test-slave-fd7l"
server_key: "a9e061689e6e2a59a0c8537054d64838"
server_key_iv: "dc84bb22c282033744de43f3547db487"
server_key_version: "encryptionkey@0"
I20260811 17:13:55.765684 23416 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:13:55.768563 23433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:55.770074 23416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 17:13:55.770259 23416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "d48ae49e8b43449a9a2cc635aab3687d"
format_stamp: "Formatted at 2026-08-11 17:13:55 on dist-test-slave-fd7l"
server_key: "a9e061689e6e2a59a0c8537054d64838"
server_key_iv: "dc84bb22c282033744de43f3547db487"
server_key_version: "encryptionkey@0"
I20260811 17:13:55.770403 23416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:13:55.791025 23416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:13:55.794993 23416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:13:55.795285 23416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:13:55.797223 23416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:13:55.797307 23416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:55.797372 23416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:13:55.797413 23416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 17:13:55 dist-test-slave-fd7l krb5kdc[22947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468435, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:13:55.809343 23416 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:36751
I20260811 17:13:55.809417 23571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:36751 every 8 connection(s)
I20260811 17:13:55.810545 23416 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:13:55.813546 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.798194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:48509 (local address 127.21.54.190:32939)
0811 17:13:55.798384 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:55.798391 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:55.799394 (+ 1003us) server_negotiation.cc:407] Connection header received
0811 17:13:55.800793 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:55.800797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:55.800868 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:55.800995 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:55.802588 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.803331 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.805016 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.805432 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.805584 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.809055 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:55.809059 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:55.809064 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:55.809125 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:55.810941 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:55.811680 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:55.811684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:55.811686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:55.811791 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:55.812234 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:55.812238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:55.812240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:55.812472 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:55.812639 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:55.813202 (+ 563us) server_negotiation.cc:299] Negotiation successful
0811 17:13:55.813349 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 17:13:55.814404 23443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.798657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:32939 (local address 127.21.54.131:48509)
0811 17:13:55.799189 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:55.799250 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:55.800468 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:55.801214 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:55.801223 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:55.801674 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:55.802378 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:55.802395 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.803484 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.803490 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:55.804843 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:55.804861 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.805635 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.805641 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:55.805872 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.806647 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:55.806667 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:55.808833 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:55.811142 (+ 2309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:55.811149 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:55.811167 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:55.811547 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:55.811913 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:55.811916 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:55.811919 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:55.812076 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:55.812639 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:55.812647 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:55.812929 (+ 282us) client_negotiation.cc:769] Sending connection context
0811 17:13:55.813133 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 17:13:55.813504 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":144,"threads_started":1}
I20260811 17:13:55.815730 23572 heartbeater.cc:347] Connected to a master server at 127.21.54.190:32939
I20260811 17:13:55.815900 23572 heartbeater.cc:464] Registering TS with master...
I20260811 17:13:55.816432 23572 heartbeater.cc:511] Master 127.21.54.190:32939 requested a full tablet report, sending...
I20260811 17:13:55.816852 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 23416
I20260811 17:13:55.816953 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:13:55.817199 21722 external_mini_cluster.cc:1428] Setting key 83ca4b42b44400738ae2795a7efc6212
I20260811 17:13:55.818990 23001 ts_manager.cc:194] Registered new tserver with Master: d48ae49e8b43449a9a2cc635aab3687d (127.21.54.131:36751)
I20260811 17:13:55.819757 23001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:48509
I20260811 17:13:55.820211 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:13:55.828392 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.818319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:47591 (local address 127.21.54.190:32939)
0811 17:13:55.818448 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:55.818452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:55.818630 (+ 178us) server_negotiation.cc:407] Connection header received
0811 17:13:55.818757 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:55.818761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:55.818813 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:55.818899 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:55.820038 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.820820 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.822032 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.822512 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.822637 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.824295 (+ 1658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:55.824298 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:55.824301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:55.824324 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:55.826262 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:55.826871 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:55.826876 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:55.826878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:55.827001 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:55.827349 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:55.827356 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:55.827360 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:55.827592 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:55.827723 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:55.828061 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 17:13:55.828192 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 17:13:55.828406 23578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.818250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:32939 (local address 127.21.54.131:47591)
0811 17:13:55.818540 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:55.818557 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:55.818653 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:55.818938 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:55.818941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:55.819172 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:55.819879 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:55.819889 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.820983 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.820987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:55.821891 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:55.821903 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.822708 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.822715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:55.822857 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.823565 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:55.823586 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:55.824118 (+ 532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:55.826483 (+ 2365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:55.826486 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:55.826488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:55.826728 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:55.827131 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:55.827137 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:55.827142 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:55.827242 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:55.827710 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:55.827714 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:55.827838 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 17:13:55.827993 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 17:13:55.828187 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":112,"threads_started":1}
Aug 11 17:13:55 dist-test-slave-fd7l krb5kdc[22947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468435, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 17:13:55 dist-test-slave-fd7l krb5kdc[22947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468435, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:13:55.852190 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.843088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44658 (local address 127.21.54.190:32939)
0811 17:13:55.843242 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:55.843246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:55.843259 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:13:55.843305 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:55.843307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:55.843350 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:55.843448 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:55.844237 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.844833 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.845735 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.845963 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.846095 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.848731 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:55.848734 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:55.848737 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:55.848764 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:55.850229 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:55.850783 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:55.850788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:55.850789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:55.850885 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:55.851185 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:55.851189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:55.851191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:55.851399 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:55.851548 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:55.851874 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 17:13:55.852018 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 17:13:55.855545 23001 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:13:55.858867 23001 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:13:55.876273 23598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.869915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:43523 (local address 127.0.0.1:47366)
0811 17:13:55.870299 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:55.870313 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:55.870398 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:55.871271 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:55.871277 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:55.871304 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:55.871394 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:55.871401 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.872905 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.872909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:55.874273 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:55.874286 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.875570 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.875574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:55.875720 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.875782 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 17:13:55.875896 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 17:13:55.876067 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":11,"thread_start_us":104,"threads_started":1}
I20260811 17:13:55.876688 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.869217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41004 (local address 127.21.54.131:36751)
0811 17:13:55.870565 (+ 1348us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:55.870578 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:55.870603 (+ 25us) server_negotiation.cc:407] Connection header received
0811 17:13:55.870682 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:55.870689 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:55.870880 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:55.871062 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:55.871372 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.872251 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.874280 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.875399 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.875543 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.875807 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:55.876356 (+ 549us) server_negotiation.cc:299] Negotiation successful
0811 17:13:55.876486 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1240,"thread_start_us":145,"threads_started":1}
I20260811 17:13:55.876847 23592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.869051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:36751 (local address 127.0.0.1:41004)
0811 17:13:55.869649 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:55.869714 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:55.869849 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:55.871047 (+ 1198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:55.871051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:55.871072 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:55.871194 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:55.871201 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.872617 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.872621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:55.874131 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:55.874142 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.875695 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.875699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:55.875964 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.876061 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 17:13:55.876132 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 17:13:55.876259 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":201,"thread_start_us":124,"threads_started":1}
I20260811 17:13:55.876868 23596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.869051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:35013 (local address 127.0.0.1:44110)
0811 17:13:55.869922 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:55.869937 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:55.870073 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:55.870824 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:55.870830 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:55.870858 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:55.871015 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:55.871022 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.872761 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.872765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:55.874273 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:55.874286 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.875695 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.875699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:55.875962 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.876037 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 17:13:55.876116 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 17:13:55.876259 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"mutex_wait_us":724,"thread_start_us":97,"threads_started":1}
I20260811 17:13:55.877573 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.870381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47366 (local address 127.21.54.129:43523)
0811 17:13:55.870714 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:55.870730 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:55.870778 (+ 48us) server_negotiation.cc:407] Connection header received
0811 17:13:55.870856 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:55.870861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:55.871074 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:55.871281 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:55.871534 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.872775 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.874420 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.875399 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.877005 (+ 1606us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.877245 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:55.877324 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 17:13:55.877441 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":150,"threads_started":1}
I20260811 17:13:55.877784 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:55.869330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44110 (local address 127.21.54.130:35013)
0811 17:13:55.870194 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:55.870210 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:55.870238 (+ 28us) server_negotiation.cc:407] Connection header received
0811 17:13:55.870325 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:55.870331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:55.870580 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:55.870801 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:55.871180 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.872612 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.874474 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:55.875515 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:55.876231 (+ 716us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:55.877466 (+ 1235us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:55.877524 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 17:13:55.877634 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":744,"thread_start_us":114,"threads_started":1}
I20260811 17:13:55.880378 23304 tablet_service.cc:1467] Processing CreateTablet for tablet 5aff25c8c56a466c970b74015c727e9c (DEFAULT_TABLE table=test-table [id=94d3747a545a4f60b667182f0e8ee3ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:13:55.880336 23163 tablet_service.cc:1467] Processing CreateTablet for tablet 5aff25c8c56a466c970b74015c727e9c (DEFAULT_TABLE table=test-table [id=94d3747a545a4f60b667182f0e8ee3ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:13:55.880376 23485 tablet_service.cc:1467] Processing CreateTablet for tablet 5aff25c8c56a466c970b74015c727e9c (DEFAULT_TABLE table=test-table [id=94d3747a545a4f60b667182f0e8ee3ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:13:55.881603 23163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5aff25c8c56a466c970b74015c727e9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:55.881623 23304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5aff25c8c56a466c970b74015c727e9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:55.881629 23485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5aff25c8c56a466c970b74015c727e9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:55.889238 23600 tablet_bootstrap.cc:492] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3: Bootstrap starting.
I20260811 17:13:55.891696 23602 tablet_bootstrap.cc:492] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72: Bootstrap starting.
I20260811 17:13:55.892828 23600 tablet_bootstrap.cc:654] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:55.892931 23601 tablet_bootstrap.cc:492] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d: Bootstrap starting.
I20260811 17:13:55.893688 23600 log.cc:824] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3: Log is configured to *not* fsync() on all Append() calls
I20260811 17:13:55.894400 23602 tablet_bootstrap.cc:654] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:55.895356 23601 tablet_bootstrap.cc:654] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:55.895448 23602 log.cc:824] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72: Log is configured to *not* fsync() on all Append() calls
I20260811 17:13:55.896173 23600 tablet_bootstrap.cc:492] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3: No bootstrap required, opened a new log
I20260811 17:13:55.896286 23601 log.cc:824] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d: Log is configured to *not* fsync() on all Append() calls
I20260811 17:13:55.896409 23600 ts_tablet_manager.cc:1397] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:13:55.898028 23602 tablet_bootstrap.cc:492] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72: No bootstrap required, opened a new log
I20260811 17:13:55.898264 23602 ts_tablet_manager.cc:1397] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 17:13:55.898737 23601 tablet_bootstrap.cc:492] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d: No bootstrap required, opened a new log
I20260811 17:13:55.898941 23601 ts_tablet_manager.cc:1397] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 17:13:55.901152 23600 raft_consensus.cc:348] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } }
I20260811 17:13:55.901427 23600 raft_consensus.cc:374] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:55.901484 23600 raft_consensus.cc:729] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 863e594fec3847d4b8315d66885846e3, State: Initialized, Role: FOLLOWER
I20260811 17:13:55.901966 23600 consensus_queue.cc:260] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } }
I20260811 17:13:55.902801 23237 heartbeater.cc:503] Master 127.21.54.190:32939 was elected leader, sending a full tablet report...
I20260811 17:13:55.903115 23600 ts_tablet_manager.cc:1428] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:13:55.902976 23602 raft_consensus.cc:348] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } }
I20260811 17:13:55.903286 23602 raft_consensus.cc:374] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:55.903350 23602 raft_consensus.cc:729] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5f7a8e1701047fcbfa410b154313c72, State: Initialized, Role: FOLLOWER
I20260811 17:13:55.903837 23602 consensus_queue.cc:260] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } }
I20260811 17:13:55.903971 23601 raft_consensus.cc:348] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } }
I20260811 17:13:55.904266 23601 raft_consensus.cc:374] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:55.904348 23601 raft_consensus.cc:729] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d48ae49e8b43449a9a2cc635aab3687d, State: Initialized, Role: FOLLOWER
I20260811 17:13:55.904778 23405 heartbeater.cc:503] Master 127.21.54.190:32939 was elected leader, sending a full tablet report...
I20260811 17:13:55.904835 23601 consensus_queue.cc:260] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } }
I20260811 17:13:55.905053 23602 ts_tablet_manager.cc:1428] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260811 17:13:55.905655 23572 heartbeater.cc:503] Master 127.21.54.190:32939 was elected leader, sending a full tablet report...
I20260811 17:13:55.906190 23601 ts_tablet_manager.cc:1428] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
W20260811 17:13:56.061702 23573 tablet.cc:2369] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:13:56.072183 23406 tablet.cc:2369] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:13:56.078179 23238 tablet.cc:2369] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:13:56.109820 23608 raft_consensus.cc:482] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:13:56.110069 23608 raft_consensus.cc:504] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } }
I20260811 17:13:56.111377 23608 leader_election.cc:288] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d48ae49e8b43449a9a2cc635aab3687d (127.21.54.131:36751), 863e594fec3847d4b8315d66885846e3 (127.21.54.129:43523)
I20260811 17:13:56.117419 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.112338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:33863 (local address 127.21.54.129:43523)
0811 17:13:56.112467 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:56.112472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:56.112488 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:13:56.112668 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:56.112674 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:56.112725 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:56.112798 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:56.113135 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.113956 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.115939 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.116680 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.116792 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.116849 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:56.117200 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 17:13:56.117278 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 17:13:56.117419 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.111884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:42243 (local address 127.21.54.131:36751)
0811 17:13:56.112105 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:56.112110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:56.112240 (+ 130us) server_negotiation.cc:407] Connection header received
0811 17:13:56.112471 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:56.112475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:56.112563 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:56.112686 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:56.113139 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.113927 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.115932 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.116617 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.116793 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.116849 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:56.117200 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 17:13:56.117277 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260811 17:13:56.117508 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.112174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:43523 (local address 127.21.54.130:33863)
0811 17:13:56.112408 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:56.112421 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:56.112530 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:56.112829 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:56.112832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:56.112846 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:56.112996 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.113002 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.114108 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.114112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.115783 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.115796 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.116804 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.116808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.117004 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.117085 (+ 81us) client_negotiation.cc:769] Sending connection context
0811 17:13:56.117164 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 17:13:56.117303 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":306,"thread_start_us":63,"threads_started":1}
I20260811 17:13:56.117762 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.111735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:36751 (local address 127.21.54.130:42243)
0811 17:13:56.112125 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:56.112143 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:56.112300 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:56.112725 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:56.112728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:56.112747 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:56.112990 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.112998 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.114108 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.114112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.115782 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.115796 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.116804 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.116808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.117007 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.117077 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 17:13:56.117164 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 17:13:56.117298 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":305,"thread_start_us":103,"threads_started":1}
I20260811 17:13:56.118341 23187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aff25c8c56a466c970b74015c727e9c" candidate_uuid: "e5f7a8e1701047fcbfa410b154313c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "863e594fec3847d4b8315d66885846e3" is_pre_election: true
I20260811 17:13:56.118474 23516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aff25c8c56a466c970b74015c727e9c" candidate_uuid: "e5f7a8e1701047fcbfa410b154313c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d48ae49e8b43449a9a2cc635aab3687d" is_pre_election: true
I20260811 17:13:56.118639 23187 raft_consensus.cc:2454] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5f7a8e1701047fcbfa410b154313c72 in term 0.
I20260811 17:13:56.118796 23516 raft_consensus.cc:2454] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5f7a8e1701047fcbfa410b154313c72 in term 0.
I20260811 17:13:56.119163 23270 leader_election.cc:302] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 863e594fec3847d4b8315d66885846e3, e5f7a8e1701047fcbfa410b154313c72; no voters:
I20260811 17:13:56.119427 23608 raft_consensus.cc:2781] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:13:56.119532 23608 raft_consensus.cc:482] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:13:56.119588 23608 raft_consensus.cc:3037] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:56.120712 23608 raft_consensus.cc:504] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } }
I20260811 17:13:56.121091 23608 leader_election.cc:288] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [CANDIDATE]: Term 1 election: Requested vote from peers d48ae49e8b43449a9a2cc635aab3687d (127.21.54.131:36751), 863e594fec3847d4b8315d66885846e3 (127.21.54.129:43523)
I20260811 17:13:56.121503 23516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aff25c8c56a466c970b74015c727e9c" candidate_uuid: "e5f7a8e1701047fcbfa410b154313c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d48ae49e8b43449a9a2cc635aab3687d"
I20260811 17:13:56.121590 23187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5aff25c8c56a466c970b74015c727e9c" candidate_uuid: "e5f7a8e1701047fcbfa410b154313c72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "863e594fec3847d4b8315d66885846e3"
I20260811 17:13:56.121670 23516 raft_consensus.cc:3037] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:56.121755 23187 raft_consensus.cc:3037] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:56.122958 23187 raft_consensus.cc:2454] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5f7a8e1701047fcbfa410b154313c72 in term 1.
I20260811 17:13:56.123358 23270 leader_election.cc:302] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 863e594fec3847d4b8315d66885846e3, e5f7a8e1701047fcbfa410b154313c72; no voters:
I20260811 17:13:56.123587 23516 raft_consensus.cc:2454] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5f7a8e1701047fcbfa410b154313c72 in term 1.
I20260811 17:13:56.123638 23608 raft_consensus.cc:2781] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:13:56.123814 23608 raft_consensus.cc:686] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [term 1 LEADER]: Becoming Leader. State: Replica: e5f7a8e1701047fcbfa410b154313c72, State: Running, Role: LEADER
I20260811 17:13:56.124248 23608 consensus_queue.cc:237] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } }
I20260811 17:13:56.127921 23001 catalog_manager.cc:5495] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 reported cstate change: term changed from 0 to 1, leader changed from <none> to e5f7a8e1701047fcbfa410b154313c72 (127.21.54.130). New cstate: current_term: 1 leader_uuid: "e5f7a8e1701047fcbfa410b154313c72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } health_report { overall_health: UNKNOWN } } }
I20260811 17:13:56.182862 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.178300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44112 (local address 127.21.54.130:35013)
0811 17:13:56.178437 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:56.178441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:56.178458 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:13:56.178648 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:56.178651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:56.178709 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:56.178814 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:13:56.179232 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.179785 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.181030 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.181256 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.181350 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.181761 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:13:56.182375 (+ 614us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:13:56.182472 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:56.182666 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 17:13:56.182737 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:13:56.190258 23516 raft_consensus.cc:1261] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d [term 1 FOLLOWER]: Refusing update from remote peer e5f7a8e1701047fcbfa410b154313c72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:13:56.191244 23608 consensus_queue.cc:1035] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:13:56.192029 23187 raft_consensus.cc:1261] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3 [term 1 FOLLOWER]: Refusing update from remote peer e5f7a8e1701047fcbfa410b154313c72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:13:56.192766 23608 consensus_queue.cc:1035] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:13:56.201745 23617 mvcc.cc:205] Tried to move back new op lower bound from 7317374714625282048 to 7317374714368974848. Current Snapshot: MvccSnapshot[applied={T|T < 7317374714625282048}]
I20260811 17:13:56.218255 23000 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:13:56.218542 23000 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:13:56.221740 23000 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524: Sending DeleteTablet for 3 replicas of tablet 5aff25c8c56a466c970b74015c727e9c
I20260811 17:13:56.222780 23304 tablet_service.cc:1514] Processing DeleteTablet for tablet 5aff25c8c56a466c970b74015c727e9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:13:56 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:44110
I20260811 17:13:56.222824 23485 tablet_service.cc:1514] Processing DeleteTablet for tablet 5aff25c8c56a466c970b74015c727e9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:13:56 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:41004
I20260811 17:13:56.222810 23163 tablet_service.cc:1514] Processing DeleteTablet for tablet 5aff25c8c56a466c970b74015c727e9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:13:56 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:47366
I20260811 17:13:56.223773 23626 tablet_replica.cc:330] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72: stopping tablet replica
I20260811 17:13:56.224061 23626 raft_consensus.cc:2229] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:13:56.224413 23626 raft_consensus.cc:2258] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:13:56.224700 23628 tablet_replica.cc:330] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3: stopping tablet replica
I20260811 17:13:56.224934 23628 raft_consensus.cc:2229] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:13:56.225260 23628 raft_consensus.cc:2258] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:13:56.226426 23000 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 17:13:56.226706 23628 ts_tablet_manager.cc:1905] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 17:13:56.226918 23000 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:13:56.229149 23627 tablet_replica.cc:330] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d: stopping tablet replica
I20260811 17:13:56.229375 23626 ts_tablet_manager.cc:1905] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:13:56.229583 23627 raft_consensus.cc:2229] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:13:56.229996 23627 raft_consensus.cc:2258] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:13:56.230446 23628 ts_tablet_manager.cc:1918] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:13:56.230576 23628 log.cc:1196] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-0/wal/wals/5aff25c8c56a466c970b74015c727e9c
I20260811 17:13:56.230947 23628 ts_tablet_manager.cc:1939] T 5aff25c8c56a466c970b74015c727e9c P 863e594fec3847d4b8315d66885846e3: Deleting consensus metadata
I20260811 17:13:56.232383 22984 catalog_manager.cc:4841] TS 863e594fec3847d4b8315d66885846e3 (127.21.54.129:43523): tablet 5aff25c8c56a466c970b74015c727e9c (table test-table [id=94d3747a545a4f60b667182f0e8ee3ef]) successfully deleted
I20260811 17:13:56.233109 23163 tablet_service.cc:1467] Processing CreateTablet for tablet 7effb79f18594d61afc51a9e225eec44 (DEFAULT_TABLE table=test-table [id=219d2b2d249a40e68ce4d521c3c1f70d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:13:56.233456 23163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7effb79f18594d61afc51a9e225eec44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:56.233439 23485 tablet_service.cc:1467] Processing CreateTablet for tablet 7effb79f18594d61afc51a9e225eec44 (DEFAULT_TABLE table=test-table [id=219d2b2d249a40e68ce4d521c3c1f70d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:13:56.233709 23485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7effb79f18594d61afc51a9e225eec44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:56.234450 23627 ts_tablet_manager.cc:1905] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:13:56.236107 23600 tablet_bootstrap.cc:492] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3: Bootstrap starting.
I20260811 17:13:56.236263 23601 tablet_bootstrap.cc:492] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d: Bootstrap starting.
I20260811 17:13:56.236968 23304 tablet_service.cc:1467] Processing CreateTablet for tablet 7effb79f18594d61afc51a9e225eec44 (DEFAULT_TABLE table=test-table [id=219d2b2d249a40e68ce4d521c3c1f70d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:13:56.237270 23304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7effb79f18594d61afc51a9e225eec44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:56.237469 23601 tablet_bootstrap.cc:654] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:56.237560 23600 tablet_bootstrap.cc:654] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:56.239324 23601 tablet_bootstrap.cc:492] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d: No bootstrap required, opened a new log
I20260811 17:13:56.239411 23601 ts_tablet_manager.cc:1397] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:13:56.239876 23601 raft_consensus.cc:348] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } }
I20260811 17:13:56.239984 23601 raft_consensus.cc:374] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:56.240025 23601 raft_consensus.cc:729] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d48ae49e8b43449a9a2cc635aab3687d, State: Initialized, Role: FOLLOWER
I20260811 17:13:56.240155 23601 consensus_queue.cc:260] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } }
I20260811 17:13:56.240433 23602 tablet_bootstrap.cc:492] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72: Bootstrap starting.
I20260811 17:13:56.240945 23600 tablet_bootstrap.cc:492] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3: No bootstrap required, opened a new log
I20260811 17:13:56.241026 23600 ts_tablet_manager.cc:1397] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 17:13:56.241333 23626 ts_tablet_manager.cc:1918] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:13:56.241428 23626 log.cc:1196] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/5aff25c8c56a466c970b74015c727e9c
I20260811 17:13:56.241529 23602 tablet_bootstrap.cc:654] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:56.241565 23600 raft_consensus.cc:348] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } }
I20260811 17:13:56.241684 23600 raft_consensus.cc:374] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:56.241727 23600 raft_consensus.cc:729] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 863e594fec3847d4b8315d66885846e3, State: Initialized, Role: FOLLOWER
I20260811 17:13:56.241788 23626 ts_tablet_manager.cc:1939] T 5aff25c8c56a466c970b74015c727e9c P e5f7a8e1701047fcbfa410b154313c72: Deleting consensus metadata
I20260811 17:13:56.241859 23600 consensus_queue.cc:260] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } }
I20260811 17:13:56.242197 23600 ts_tablet_manager.cc:1428] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 17:13:56.243222 23602 tablet_bootstrap.cc:492] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72: No bootstrap required, opened a new log
I20260811 17:13:56.243305 23602 ts_tablet_manager.cc:1397] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260811 17:13:56.243253 22983 catalog_manager.cc:4841] TS e5f7a8e1701047fcbfa410b154313c72 (127.21.54.130:35013): tablet 5aff25c8c56a466c970b74015c727e9c (table test-table [id=94d3747a545a4f60b667182f0e8ee3ef]) successfully deleted
I20260811 17:13:56.243530 23601 ts_tablet_manager.cc:1428] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260811 17:13:56.244097 23602 raft_consensus.cc:348] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } }
I20260811 17:13:56.244206 23602 raft_consensus.cc:374] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:56.244236 23602 raft_consensus.cc:729] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5f7a8e1701047fcbfa410b154313c72, State: Initialized, Role: FOLLOWER
I20260811 17:13:56.245014 23602 consensus_queue.cc:260] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } }
I20260811 17:13:56.245782 23602 ts_tablet_manager.cc:1428] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 17:13:56.245828 23627 ts_tablet_manager.cc:1918] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:13:56.245939 23627 log.cc:1196] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/5aff25c8c56a466c970b74015c727e9c
I20260811 17:13:56.246325 23627 ts_tablet_manager.cc:1939] T 5aff25c8c56a466c970b74015c727e9c P d48ae49e8b43449a9a2cc635aab3687d: Deleting consensus metadata
I20260811 17:13:56.247943 22984 catalog_manager.cc:4841] TS d48ae49e8b43449a9a2cc635aab3687d (127.21.54.131:36751): tablet 5aff25c8c56a466c970b74015c727e9c (table test-table [id=94d3747a545a4f60b667182f0e8ee3ef]) successfully deleted
I20260811 17:13:56.318167 23609 raft_consensus.cc:482] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:13:56.318375 23609 raft_consensus.cc:504] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } }
I20260811 17:13:56.319695 23609 leader_election.cc:288] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 863e594fec3847d4b8315d66885846e3 (127.21.54.129:43523), e5f7a8e1701047fcbfa410b154313c72 (127.21.54.130:35013)
I20260811 17:13:56.324824 23633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.319939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:35013 (local address 127.21.54.131:53309)
0811 17:13:56.320737 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:56.320751 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:56.320959 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:56.321328 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:56.321333 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:56.321352 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:56.321455 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.321461 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.322366 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.322372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.323359 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.323370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.324264 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.324268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.324367 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.324396 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 17:13:56.324554 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 17:13:56.324643 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"mutex_wait_us":435,"thread_start_us":84,"threads_started":1}
I20260811 17:13:56.324822 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.320199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:53309 (local address 127.21.54.130:35013)
0811 17:13:56.320336 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:56.320340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:56.320877 (+ 537us) server_negotiation.cc:407] Connection header received
0811 17:13:56.321091 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:56.321094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:56.321147 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:56.321192 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:56.321590 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.322252 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.323498 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.324117 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.324247 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.324323 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:56.324593 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 17:13:56.324695 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:13:56.325172 23632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.319940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:43523 (local address 127.21.54.131:56053)
0811 17:13:56.320306 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:56.320321 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:56.320542 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:56.320956 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:56.320971 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:56.320993 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:56.321125 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.321131 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.322088 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.322093 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.323505 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.323516 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.324421 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.324425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.324578 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.324604 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 17:13:56.324890 (+ 286us) client_negotiation.cc:240] Negotiation successful
0811 17:13:56.324993 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":22,"thread_start_us":91,"threads_started":1}
I20260811 17:13:56.325215 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.320040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:56053 (local address 127.21.54.129:43523)
0811 17:13:56.320265 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:56.320270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:56.320438 (+ 168us) server_negotiation.cc:407] Connection header received
0811 17:13:56.320679 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:56.320682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:56.320736 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:56.320793 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:56.321252 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.321941 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.323641 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.324259 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.324377 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.324430 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:56.324730 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 17:13:56.324809 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260811 17:13:56.325624 23346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7effb79f18594d61afc51a9e225eec44" candidate_uuid: "d48ae49e8b43449a9a2cc635aab3687d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5f7a8e1701047fcbfa410b154313c72" is_pre_election: true
I20260811 17:13:56.325685 23187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7effb79f18594d61afc51a9e225eec44" candidate_uuid: "d48ae49e8b43449a9a2cc635aab3687d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "863e594fec3847d4b8315d66885846e3" is_pre_election: true
I20260811 17:13:56.325834 23187 raft_consensus.cc:2454] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d48ae49e8b43449a9a2cc635aab3687d in term 0.
I20260811 17:13:56.325888 23346 raft_consensus.cc:2454] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d48ae49e8b43449a9a2cc635aab3687d in term 0.
I20260811 17:13:56.326297 23435 leader_election.cc:302] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 863e594fec3847d4b8315d66885846e3, d48ae49e8b43449a9a2cc635aab3687d, e5f7a8e1701047fcbfa410b154313c72; no voters:
I20260811 17:13:56.326694 23609 raft_consensus.cc:2781] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:13:56.326756 23609 raft_consensus.cc:482] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:13:56.326797 23609 raft_consensus.cc:3037] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:56.327838 23609 raft_consensus.cc:504] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } }
I20260811 17:13:56.328226 23609 leader_election.cc:288] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [CANDIDATE]: Term 1 election: Requested vote from peers 863e594fec3847d4b8315d66885846e3 (127.21.54.129:43523), e5f7a8e1701047fcbfa410b154313c72 (127.21.54.130:35013)
I20260811 17:13:56.328639 23187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7effb79f18594d61afc51a9e225eec44" candidate_uuid: "d48ae49e8b43449a9a2cc635aab3687d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "863e594fec3847d4b8315d66885846e3"
I20260811 17:13:56.328773 23187 raft_consensus.cc:3037] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:56.329444 23346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7effb79f18594d61afc51a9e225eec44" candidate_uuid: "d48ae49e8b43449a9a2cc635aab3687d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5f7a8e1701047fcbfa410b154313c72"
I20260811 17:13:56.329574 23346 raft_consensus.cc:3037] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:56.330132 23187 raft_consensus.cc:2454] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d48ae49e8b43449a9a2cc635aab3687d in term 1.
I20260811 17:13:56.330742 23435 leader_election.cc:302] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 863e594fec3847d4b8315d66885846e3, d48ae49e8b43449a9a2cc635aab3687d; no voters:
I20260811 17:13:56.330919 23346 raft_consensus.cc:2454] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d48ae49e8b43449a9a2cc635aab3687d in term 1.
I20260811 17:13:56.330971 23609 raft_consensus.cc:2781] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:13:56.331143 23609 raft_consensus.cc:686] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [term 1 LEADER]: Becoming Leader. State: Replica: d48ae49e8b43449a9a2cc635aab3687d, State: Running, Role: LEADER
I20260811 17:13:56.331420 23609 consensus_queue.cc:237] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } }
I20260811 17:13:56.334223 23000 catalog_manager.cc:5495] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d reported cstate change: term changed from 0 to 1, leader changed from <none> to d48ae49e8b43449a9a2cc635aab3687d (127.21.54.131). New cstate: current_term: 1 leader_uuid: "d48ae49e8b43449a9a2cc635aab3687d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } health_report { overall_health: HEALTHY } } }
Aug 11 17:13:56 dist-test-slave-fd7l krb5kdc[22947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468434, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:13:56.377566 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.366462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:32939 (local address 127.0.0.1:44668)
0811 17:13:56.366904 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:56.366922 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:56.367071 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:56.367449 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:56.367453 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:56.368007 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:56.368147 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.368156 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.369114 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.369118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.370018 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.370029 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.370557 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.370561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.370649 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.371267 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:56.371282 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:56.373265 (+ 1983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:56.375557 (+ 2292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:56.375570 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:56.375577 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:56.375903 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:56.376309 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:56.376312 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:56.376314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:56.376411 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:56.376899 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:56.376908 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:56.377047 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 17:13:56.377189 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 17:13:56.377359 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":143,"threads_started":1}
I20260811 17:13:56.377600 23645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.366586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44668 (local address 127.21.54.190:32939)
0811 17:13:56.366876 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:56.366885 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:56.367022 (+ 137us) server_negotiation.cc:407] Connection header received
0811 17:13:56.367214 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:56.367216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:56.367262 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:56.367361 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:56.368295 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.368998 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.370149 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.370388 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.370527 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.373491 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:56.373494 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:56.373496 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:56.373522 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:56.375440 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:56.376082 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:56.376085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:56.376086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:56.376173 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:56.376572 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:56.376578 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:56.376582 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:56.376785 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:56.376899 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:56.377364 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 17:13:56.377481 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":80,"threads_started":1}
I20260811 17:13:56.382169 23000 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:44668:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 17:13:56.383038 23000 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:13:56.389292 23485 tablet_service.cc:1467] Processing CreateTablet for tablet 1eac782eabd1438db4d20dc8ef61d40e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8e6ca00181cf469d89ef03d4e850e5a7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 17:13:56.389755 23485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eac782eabd1438db4d20dc8ef61d40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:56.389272 23304 tablet_service.cc:1467] Processing CreateTablet for tablet 1eac782eabd1438db4d20dc8ef61d40e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8e6ca00181cf469d89ef03d4e850e5a7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 17:13:56.392298 23304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eac782eabd1438db4d20dc8ef61d40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:56.392299 23601 tablet_bootstrap.cc:492] T 1eac782eabd1438db4d20dc8ef61d40e P d48ae49e8b43449a9a2cc635aab3687d: Bootstrap starting.
I20260811 17:13:56.393373 23601 tablet_bootstrap.cc:654] T 1eac782eabd1438db4d20dc8ef61d40e P d48ae49e8b43449a9a2cc635aab3687d: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:56.393883 23163 tablet_service.cc:1467] Processing CreateTablet for tablet 1eac782eabd1438db4d20dc8ef61d40e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8e6ca00181cf469d89ef03d4e850e5a7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 17:13:56.394302 23163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eac782eabd1438db4d20dc8ef61d40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:56.394850 23601 tablet_bootstrap.cc:492] T 1eac782eabd1438db4d20dc8ef61d40e P d48ae49e8b43449a9a2cc635aab3687d: No bootstrap required, opened a new log
I20260811 17:13:56.394953 23601 ts_tablet_manager.cc:1397] T 1eac782eabd1438db4d20dc8ef61d40e P d48ae49e8b43449a9a2cc635aab3687d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:13:56.395421 23601 raft_consensus.cc:348] T 1eac782eabd1438db4d20dc8ef61d40e P d48ae49e8b43449a9a2cc635aab3687d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } }
I20260811 17:13:56.395556 23601 raft_consensus.cc:374] T 1eac782eabd1438db4d20dc8ef61d40e P d48ae49e8b43449a9a2cc635aab3687d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:56.395596 23601 raft_consensus.cc:729] T 1eac782eabd1438db4d20dc8ef61d40e P d48ae49e8b43449a9a2cc635aab3687d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d48ae49e8b43449a9a2cc635aab3687d, State: Initialized, Role: FOLLOWER
I20260811 17:13:56.395741 23601 consensus_queue.cc:260] T 1eac782eabd1438db4d20dc8ef61d40e P d48ae49e8b43449a9a2cc635aab3687d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } }
I20260811 17:13:56.396055 23601 ts_tablet_manager.cc:1428] T 1eac782eabd1438db4d20dc8ef61d40e P d48ae49e8b43449a9a2cc635aab3687d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:13:56.396277 23602 tablet_bootstrap.cc:492] T 1eac782eabd1438db4d20dc8ef61d40e P e5f7a8e1701047fcbfa410b154313c72: Bootstrap starting.
I20260811 17:13:56.396390 23609 tablet_replica.cc:439] T 1eac782eabd1438db4d20dc8ef61d40e P d48ae49e8b43449a9a2cc635aab3687d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } } }
I20260811 17:13:56.396544 23609 tablet_replica.cc:442] T 1eac782eabd1438db4d20dc8ef61d40e P d48ae49e8b43449a9a2cc635aab3687d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:13:56.397226 23602 tablet_bootstrap.cc:654] T 1eac782eabd1438db4d20dc8ef61d40e P e5f7a8e1701047fcbfa410b154313c72: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:56.397606 23600 tablet_bootstrap.cc:492] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3: Bootstrap starting.
I20260811 17:13:56.398494 23602 tablet_bootstrap.cc:492] T 1eac782eabd1438db4d20dc8ef61d40e P e5f7a8e1701047fcbfa410b154313c72: No bootstrap required, opened a new log
I20260811 17:13:56.398581 23600 tablet_bootstrap.cc:654] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:56.398592 23602 ts_tablet_manager.cc:1397] T 1eac782eabd1438db4d20dc8ef61d40e P e5f7a8e1701047fcbfa410b154313c72: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 17:13:56.398996 23602 raft_consensus.cc:348] T 1eac782eabd1438db4d20dc8ef61d40e P e5f7a8e1701047fcbfa410b154313c72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } }
I20260811 17:13:56.399135 23602 raft_consensus.cc:374] T 1eac782eabd1438db4d20dc8ef61d40e P e5f7a8e1701047fcbfa410b154313c72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:56.399184 23602 raft_consensus.cc:729] T 1eac782eabd1438db4d20dc8ef61d40e P e5f7a8e1701047fcbfa410b154313c72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5f7a8e1701047fcbfa410b154313c72, State: Initialized, Role: FOLLOWER
I20260811 17:13:56.399348 23602 consensus_queue.cc:260] T 1eac782eabd1438db4d20dc8ef61d40e P e5f7a8e1701047fcbfa410b154313c72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } }
I20260811 17:13:56.399624 23602 ts_tablet_manager.cc:1428] T 1eac782eabd1438db4d20dc8ef61d40e P e5f7a8e1701047fcbfa410b154313c72: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:13:56.399852 23600 tablet_bootstrap.cc:492] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3: No bootstrap required, opened a new log
I20260811 17:13:56.399842 23620 tablet_replica.cc:439] T 1eac782eabd1438db4d20dc8ef61d40e P e5f7a8e1701047fcbfa410b154313c72: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } } }
I20260811 17:13:56.399955 23600 ts_tablet_manager.cc:1397] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 17:13:56.399991 23620 tablet_replica.cc:442] T 1eac782eabd1438db4d20dc8ef61d40e P e5f7a8e1701047fcbfa410b154313c72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:13:56.400420 23600 raft_consensus.cc:348] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } }
I20260811 17:13:56.400579 23600 raft_consensus.cc:374] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:56.400615 23600 raft_consensus.cc:729] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 863e594fec3847d4b8315d66885846e3, State: Initialized, Role: FOLLOWER
I20260811 17:13:56.400736 23600 consensus_queue.cc:260] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } }
I20260811 17:13:56.401010 23607 tablet_replica.cc:439] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } } }
I20260811 17:13:56.401197 23607 tablet_replica.cc:442] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:13:56.401517 23600 ts_tablet_manager.cc:1428] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 17:13:56.604758 23607 raft_consensus.cc:482] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:13:56.604926 23607 raft_consensus.cc:504] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } }
I20260811 17:13:56.606227 23607 leader_election.cc:288] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5f7a8e1701047fcbfa410b154313c72 (127.21.54.130:35013), d48ae49e8b43449a9a2cc635aab3687d (127.21.54.131:36751)
I20260811 17:13:56.610466 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.606678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:32981 (local address 127.21.54.130:35013)
0811 17:13:56.606824 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:56.606828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:56.607056 (+ 228us) server_negotiation.cc:407] Connection header received
0811 17:13:56.607207 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:56.607216 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:56.607260 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:56.607350 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:56.607640 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.608161 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.609380 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.609788 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.609927 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.609989 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:56.610243 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 17:13:56.610330 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 17:13:56.610483 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.606558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:35013 (local address 127.21.54.129:32981)
0811 17:13:56.606933 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:56.606956 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:56.607076 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:56.607365 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:56.607368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:56.607385 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:56.607513 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.607525 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.608277 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.608283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.609239 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.609249 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.609927 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.609931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.610101 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.610126 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 17:13:56.610222 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 17:13:56.610302 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":31,"thread_start_us":141,"threads_started":1}
I20260811 17:13:56.610483 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.606714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:38423 (local address 127.21.54.131:36751)
0811 17:13:56.606965 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:56.606971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:56.607109 (+ 138us) server_negotiation.cc:407] Connection header received
0811 17:13:56.607324 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:56.607327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:56.607368 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:56.607523 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:56.607772 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.608293 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.609425 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.609827 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.609971 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.610027 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:56.610268 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 17:13:56.610349 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260811 17:13:56.610510 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.606558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:36751 (local address 127.21.54.129:38423)
0811 17:13:56.607020 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:56.607038 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:56.607177 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:56.607475 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:56.607478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:56.607496 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:56.607658 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.607664 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.608432 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.608435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.609284 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.609295 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.609971 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.609974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.610135 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.610153 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 17:13:56.610248 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 17:13:56.610324 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":72,"thread_start_us":124,"threads_started":1}
I20260811 17:13:56.611294 23516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eac782eabd1438db4d20dc8ef61d40e" candidate_uuid: "863e594fec3847d4b8315d66885846e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d48ae49e8b43449a9a2cc635aab3687d" is_pre_election: true
I20260811 17:13:56.611339 23346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eac782eabd1438db4d20dc8ef61d40e" candidate_uuid: "863e594fec3847d4b8315d66885846e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5f7a8e1701047fcbfa410b154313c72" is_pre_election: true
I20260811 17:13:56.611485 23516 raft_consensus.cc:2454] T 1eac782eabd1438db4d20dc8ef61d40e P d48ae49e8b43449a9a2cc635aab3687d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 863e594fec3847d4b8315d66885846e3 in term 0.
I20260811 17:13:56.611500 23346 raft_consensus.cc:2454] T 1eac782eabd1438db4d20dc8ef61d40e P e5f7a8e1701047fcbfa410b154313c72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 863e594fec3847d4b8315d66885846e3 in term 0.
I20260811 17:13:56.611964 23103 leader_election.cc:302] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 863e594fec3847d4b8315d66885846e3, d48ae49e8b43449a9a2cc635aab3687d, e5f7a8e1701047fcbfa410b154313c72; no voters:
I20260811 17:13:56.612253 23607 raft_consensus.cc:2781] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:13:56.612310 23607 raft_consensus.cc:482] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:13:56.612337 23607 raft_consensus.cc:3037] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:56.613706 23607 raft_consensus.cc:504] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } }
I20260811 17:13:56.614182 23607 leader_election.cc:288] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 [CANDIDATE]: Term 1 election: Requested vote from peers e5f7a8e1701047fcbfa410b154313c72 (127.21.54.130:35013), d48ae49e8b43449a9a2cc635aab3687d (127.21.54.131:36751)
I20260811 17:13:56.614637 23346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eac782eabd1438db4d20dc8ef61d40e" candidate_uuid: "863e594fec3847d4b8315d66885846e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5f7a8e1701047fcbfa410b154313c72"
I20260811 17:13:56.614661 23516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eac782eabd1438db4d20dc8ef61d40e" candidate_uuid: "863e594fec3847d4b8315d66885846e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d48ae49e8b43449a9a2cc635aab3687d"
I20260811 17:13:56.614779 23346 raft_consensus.cc:3037] T 1eac782eabd1438db4d20dc8ef61d40e P e5f7a8e1701047fcbfa410b154313c72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:56.614791 23516 raft_consensus.cc:3037] T 1eac782eabd1438db4d20dc8ef61d40e P d48ae49e8b43449a9a2cc635aab3687d [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:56.615839 23516 raft_consensus.cc:2454] T 1eac782eabd1438db4d20dc8ef61d40e P d48ae49e8b43449a9a2cc635aab3687d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 863e594fec3847d4b8315d66885846e3 in term 1.
I20260811 17:13:56.615839 23346 raft_consensus.cc:2454] T 1eac782eabd1438db4d20dc8ef61d40e P e5f7a8e1701047fcbfa410b154313c72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 863e594fec3847d4b8315d66885846e3 in term 1.
I20260811 17:13:56.616142 23103 leader_election.cc:302] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 863e594fec3847d4b8315d66885846e3, e5f7a8e1701047fcbfa410b154313c72; no voters:
I20260811 17:13:56.616344 23607 raft_consensus.cc:2781] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:13:56.616664 23607 raft_consensus.cc:686] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 [term 1 LEADER]: Becoming Leader. State: Replica: 863e594fec3847d4b8315d66885846e3, State: Running, Role: LEADER
I20260811 17:13:56.616989 23607 consensus_queue.cc:237] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } }
I20260811 17:13:56.618641 23651 tablet_replica.cc:439] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3: TxnStatusTablet state changed. Reason: New leader 863e594fec3847d4b8315d66885846e3. Latest consensus state: current_term: 1 leader_uuid: "863e594fec3847d4b8315d66885846e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } } }
I20260811 17:13:56.618758 23651 tablet_replica.cc:442] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3: This TxnStatusTablet replica's current role is: LEADER
I20260811 17:13:56.619133 23654 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 17:13:56.619364 23654 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 17:13:56.619942 23000 catalog_manager.cc:5495] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 863e594fec3847d4b8315d66885846e3 (127.21.54.129). New cstate: current_term: 1 leader_uuid: "863e594fec3847d4b8315d66885846e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } health_report { overall_health: UNKNOWN } } }
I20260811 17:13:56.644203 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.639268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:43523 (local address 127.0.0.1:47370)
0811 17:13:56.639417 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:56.639434 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:56.639550 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:56.639927 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:56.639929 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:56.639936 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:13:56.640060 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.640065 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.641178 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.641181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.642020 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.642031 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.642693 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.642698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.642832 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.642889 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:13:56.643884 (+ 995us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:13:56.643908 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 17:13:56.643982 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 17:13:56.644079 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 17:13:56.644208 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.639368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47370 (local address 127.21.54.129:43523)
0811 17:13:56.639507 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:56.639511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:56.639528 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:13:56.639714 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:56.639717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:56.639786 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:56.639904 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:13:56.640170 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.641035 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.642153 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.642538 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.642645 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.643079 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:13:56.643775 (+ 696us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:13:56.643911 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:56.643997 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 17:13:56.644061 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 17:13:56.647631 23516 raft_consensus.cc:1261] T 1eac782eabd1438db4d20dc8ef61d40e P d48ae49e8b43449a9a2cc635aab3687d [term 1 FOLLOWER]: Refusing update from remote peer 863e594fec3847d4b8315d66885846e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:13:56.647778 23346 raft_consensus.cc:1261] T 1eac782eabd1438db4d20dc8ef61d40e P e5f7a8e1701047fcbfa410b154313c72 [term 1 FOLLOWER]: Refusing update from remote peer 863e594fec3847d4b8315d66885846e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:13:56.648178 23651 consensus_queue.cc:1035] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:13:56.648386 23607 consensus_queue.cc:1035] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:13:56.650157 23620 tablet_replica.cc:439] T 1eac782eabd1438db4d20dc8ef61d40e P e5f7a8e1701047fcbfa410b154313c72: TxnStatusTablet state changed. Reason: New leader 863e594fec3847d4b8315d66885846e3. Latest consensus state: current_term: 1 leader_uuid: "863e594fec3847d4b8315d66885846e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } } }
I20260811 17:13:56.650161 23609 tablet_replica.cc:439] T 1eac782eabd1438db4d20dc8ef61d40e P d48ae49e8b43449a9a2cc635aab3687d: TxnStatusTablet state changed. Reason: New leader 863e594fec3847d4b8315d66885846e3. Latest consensus state: current_term: 1 leader_uuid: "863e594fec3847d4b8315d66885846e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } } }
I20260811 17:13:56.650274 23620 tablet_replica.cc:442] T 1eac782eabd1438db4d20dc8ef61d40e P e5f7a8e1701047fcbfa410b154313c72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:13:56.650274 23609 tablet_replica.cc:442] T 1eac782eabd1438db4d20dc8ef61d40e P d48ae49e8b43449a9a2cc635aab3687d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:13:56.652866 23620 tablet_replica.cc:439] T 1eac782eabd1438db4d20dc8ef61d40e P e5f7a8e1701047fcbfa410b154313c72: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "863e594fec3847d4b8315d66885846e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } } }
I20260811 17:13:56.652998 23620 tablet_replica.cc:442] T 1eac782eabd1438db4d20dc8ef61d40e P e5f7a8e1701047fcbfa410b154313c72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:13:56.653870 23651 tablet_replica.cc:439] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "863e594fec3847d4b8315d66885846e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } } }
I20260811 17:13:56.654080 23651 tablet_replica.cc:442] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3: This TxnStatusTablet replica's current role is: LEADER
I20260811 17:13:56.654663 23615 mvcc.cc:205] Tried to move back new op lower bound from 7317374716503007232 to 7317374716385886208. Current Snapshot: MvccSnapshot[applied={T|T < 7317374716503007232 or (T in {7317374716503007232})}]
I20260811 17:13:56.655799 23651 tablet_replica.cc:439] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "863e594fec3847d4b8315d66885846e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } } }
I20260811 17:13:56.656054 23651 tablet_replica.cc:442] T 1eac782eabd1438db4d20dc8ef61d40e P 863e594fec3847d4b8315d66885846e3: This TxnStatusTablet replica's current role is: LEADER
I20260811 17:13:56.657122 23616 mvcc.cc:205] Tried to move back new op lower bound from 7317374716503007232 to 7317374716385886208. Current Snapshot: MvccSnapshot[applied={T|T < 7317374716503007232}]
I20260811 17:13:56.657516 23609 tablet_replica.cc:439] T 1eac782eabd1438db4d20dc8ef61d40e P d48ae49e8b43449a9a2cc635aab3687d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "863e594fec3847d4b8315d66885846e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 } } peers { permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 } } peers { permanent_uuid: "d48ae49e8b43449a9a2cc635aab3687d" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36751 } } }
I20260811 17:13:56.657627 23609 tablet_replica.cc:442] T 1eac782eabd1438db4d20dc8ef61d40e P d48ae49e8b43449a9a2cc635aab3687d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:13:56.663874 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.659738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41006 (local address 127.21.54.131:36751)
0811 17:13:56.659865 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:56.659870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:56.659883 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:13:56.659960 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:56.659964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:56.660038 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:56.660128 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:13:56.660591 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.661148 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.662334 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.662557 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.662672 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.662964 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:13:56.663431 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:13:56.663520 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:56.663648 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 17:13:56.663701 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:13:56.676192 23663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.671860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:43523 (local address 127.21.54.131:51345)
0811 17:13:56.672200 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:56.672217 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:56.672373 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:56.672723 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:56.672726 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:56.672734 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:13:56.672856 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.672861 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.673603 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.673607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.674842 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.674854 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.675375 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.675379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.675486 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.675539 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:13:56.675859 (+ 320us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:13:56.675868 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 17:13:56.675935 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 17:13:56.676021 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":119,"threads_started":1}
I20260811 17:13:56.676177 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.671981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:51345 (local address 127.21.54.129:43523)
0811 17:13:56.672201 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:56.672207 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:56.672307 (+ 100us) server_negotiation.cc:407] Connection header received
0811 17:13:56.672490 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:56.672493 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:56.672604 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:56.672702 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:13:56.672973 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.673484 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.674969 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.675209 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.675351 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.675685 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:13:56.675773 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:13:56.675854 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:56.675965 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 17:13:56.676019 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 17:13:56.684494 23346 raft_consensus.cc:1261] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72 [term 1 FOLLOWER]: Refusing update from remote peer d48ae49e8b43449a9a2cc635aab3687d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:13:56.684494 23187 raft_consensus.cc:1261] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3 [term 1 FOLLOWER]: Refusing update from remote peer d48ae49e8b43449a9a2cc635aab3687d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:13:56.685066 23609 consensus_queue.cc:1035] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5f7a8e1701047fcbfa410b154313c72" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 35013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:13:56.685334 23634 consensus_queue.cc:1035] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [LEADER]: Connected to new peer: Peer: permanent_uuid: "863e594fec3847d4b8315d66885846e3" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 43523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:13:56.715312 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.710675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:47429 (local address 127.21.54.131:36751)
0811 17:13:56.710812 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:56.710817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:56.711165 (+ 348us) server_negotiation.cc:407] Connection header received
0811 17:13:56.711353 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:56.711356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:56.711426 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:56.711508 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:13:56.711816 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.712654 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.713967 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.714313 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.714415 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.714864 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:13:56.714950 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:13:56.715027 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:56.715090 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 17:13:56.715146 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 17:13:56.715354 23668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.710618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:36751 (local address 127.21.54.129:47429)
0811 17:13:56.711049 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:56.711070 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:56.711219 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:56.711539 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:56.711542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:56.711564 (+ 22us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:13:56.711690 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.711695 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.712779 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.712783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.713818 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:56.713830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.714536 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.714542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:56.714654 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.714723 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:13:56.715029 (+ 306us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:13:56.715033 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 17:13:56.715080 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 17:13:56.715152 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":129,"threads_started":1}
I20260811 17:13:56.745163 23002 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:13:56.745361 23002 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:13:56.748107 23002 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6bf63d35a6004a8491b53136f51f2524: Sending DeleteTablet for 3 replicas of tablet 7effb79f18594d61afc51a9e225eec44
I20260811 17:13:56.749114 23163 tablet_service.cc:1514] Processing DeleteTablet for tablet 7effb79f18594d61afc51a9e225eec44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:13:56 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:47366
I20260811 17:13:56.749383 23485 tablet_service.cc:1514] Processing DeleteTablet for tablet 7effb79f18594d61afc51a9e225eec44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:13:56 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:41004
I20260811 17:13:56.749454 23671 tablet_replica.cc:330] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3: stopping tablet replica
I20260811 17:13:56.749596 23671 raft_consensus.cc:2229] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:13:56.749668 23304 tablet_service.cc:1514] Processing DeleteTablet for tablet 7effb79f18594d61afc51a9e225eec44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:13:56 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:44110
I20260811 17:13:56.749722 23671 raft_consensus.cc:2258] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:13:56.750490 23671 ts_tablet_manager.cc:1905] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:13:56.751639 23672 tablet_replica.cc:330] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d: stopping tablet replica
I20260811 17:13:56.751866 23673 tablet_replica.cc:330] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72: stopping tablet replica
I20260811 17:13:56.752048 23673 raft_consensus.cc:2229] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:13:56.752216 23672 raft_consensus.cc:2229] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:13:56.752411 23673 raft_consensus.cc:2258] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:13:56.752869 23672 raft_consensus.cc:2258] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:13:56.754243 23671 ts_tablet_manager.cc:1918] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:13:56.754366 23673 ts_tablet_manager.cc:1905] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:13:56.754619 23671 log.cc:1196] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-0/wal/wals/7effb79f18594d61afc51a9e225eec44
I20260811 17:13:56.754994 23671 ts_tablet_manager.cc:1939] T 7effb79f18594d61afc51a9e225eec44 P 863e594fec3847d4b8315d66885846e3: Deleting consensus metadata
I20260811 17:13:56.755100 23672 ts_tablet_manager.cc:1905] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:13:56.755848 22984 catalog_manager.cc:4841] TS 863e594fec3847d4b8315d66885846e3 (127.21.54.129:43523): tablet 7effb79f18594d61afc51a9e225eec44 (table test-table [id=219d2b2d249a40e68ce4d521c3c1f70d]) successfully deleted
I20260811 17:13:56.758344 23673 ts_tablet_manager.cc:1918] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:13:56.758450 23673 log.cc:1196] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/7effb79f18594d61afc51a9e225eec44
I20260811 17:13:56.760874 23673 ts_tablet_manager.cc:1939] T 7effb79f18594d61afc51a9e225eec44 P e5f7a8e1701047fcbfa410b154313c72: Deleting consensus metadata
I20260811 17:13:56.761533 23672 ts_tablet_manager.cc:1918] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:13:56.761674 23672 log.cc:1196] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/7effb79f18594d61afc51a9e225eec44
I20260811 17:13:56.761785 22983 catalog_manager.cc:4841] TS e5f7a8e1701047fcbfa410b154313c72 (127.21.54.130:35013): tablet 7effb79f18594d61afc51a9e225eec44 (table test-table [id=219d2b2d249a40e68ce4d521c3c1f70d]) successfully deleted
I20260811 17:13:56.762013 23672 ts_tablet_manager.cc:1939] T 7effb79f18594d61afc51a9e225eec44 P d48ae49e8b43449a9a2cc635aab3687d: Deleting consensus metadata
I20260811 17:13:56.762854 22984 catalog_manager.cc:4841] TS d48ae49e8b43449a9a2cc635aab3687d (127.21.54.131:36751): tablet 7effb79f18594d61afc51a9e225eec44 (table test-table [id=219d2b2d249a40e68ce4d521c3c1f70d]) successfully deleted
Aug 11 17:13:56 dist-test-slave-fd7l krb5kdc[22947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468435, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.54.129@KRBTEST.COM
I20260811 17:13:56.780144 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.753558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47380 (local address 127.21.54.129:43523)
0811 17:13:56.753678 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:56.753684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:56.763358 (+ 9674us) server_negotiation.cc:407] Connection header received
0811 17:13:56.763497 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:56.763501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:56.763563 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:56.763651 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:56.764580 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.765412 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.766702 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.767047 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.767174 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.775316 (+ 8142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:56.775326 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:56.775346 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:56.775390 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:56.777906 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:56.778500 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:56.778508 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:56.778510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:56.778631 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:56.778976 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:56.778979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:56.778981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:56.779350 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:56.779624 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:56.779809 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 17:13:56.779955 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 17:13:56.781127 23231 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47380
I20260811 17:13:56.792933 23645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:56.783140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44672 (local address 127.21.54.190:32939)
0811 17:13:56.783271 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:56.783276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:56.783744 (+ 468us) server_negotiation.cc:407] Connection header received
0811 17:13:56.783844 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:56.783848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:56.783907 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:56.784004 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:56.784974 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.785800 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.786951 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:56.787302 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:56.787405 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:56.789142 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:56.789146 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:56.789149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:56.789180 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:56.790915 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:56.791462 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:56.791466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:56.791473 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:56.791603 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:56.791914 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:56.791917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:56.791919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:56.792151 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:56.792295 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:56.792599 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 17:13:56.792744 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 17:13:56.793742 23002 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44672
I20260811 17:13:56.795827 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 23084
I20260811 17:13:56.810065 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 23250
I20260811 17:13:56.826244 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 23416
I20260811 17:13:56.834985 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 22962
2026-08-11T17:13:56Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3323 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:13:56 dist-test-slave-fd7l krb5kdc[23682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:13:56 dist-test-slave-fd7l krb5kdc[23682](info): set up 2 sockets
Aug 11 17:13:56 dist-test-slave-fd7l krb5kdc[23682](info): commencing operation
krb5kdc: starting...
W20260811 17:13:57.975880 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:13:58 dist-test-slave-fd7l krb5kdc[23682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:13:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:13:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:13:58.126252 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:35417
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:35417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:13:58.251181 23696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:13:58.251525 23696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:13:58.251571 23696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:13:58.255832 23696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:13:58.255920 23696 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 17:13:58.255939 23696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:13:58.255957 23696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:13:58.255981 23696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:13:58.255996 23696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:13:58.261921 23696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:35417
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:35417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:13:58.263199 23696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:13:58.264320 23696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:13:58.274127 23704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:58.274271 23696 server_base.cc:1034] running on GCE node
W20260811 17:13:58.274139 23702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:58.274139 23701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:58.275197 23696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:13:58.276482 23696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:13:58.277786 23696 hybrid_clock.cc:648] HybridClock initialized: now 1786468438277766 us; error 35 us; skew 500 ppm
Aug 11 17:13:58 dist-test-slave-fd7l krb5kdc[23682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468438, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:13:58.287525 23696 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:13:58.288854 23696 webserver.cc:460] Webserver started at http://127.21.54.190:46113/ using document root <none> and password file <none>
I20260811 17:13:58.289538 23696 fs_manager.cc:359] Metadata directory not provided
I20260811 17:13:58.289654 23696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:13:58.289928 23696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:13:58.292282 23696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "01a1ecf48bad4bb3ac29277ac50264c2"
format_stamp: "Formatted at 2026-08-11 17:13:58 on dist-test-slave-fd7l"
server_key: "aeb2c437a5ececb05aa62a4eba581ecb"
server_key_iv: "734606bd8f60845c1fa8165c1edd4adc"
server_key_version: "encryptionkey@0"
I20260811 17:13:58.292989 23696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "01a1ecf48bad4bb3ac29277ac50264c2"
format_stamp: "Formatted at 2026-08-11 17:13:58 on dist-test-slave-fd7l"
server_key: "aeb2c437a5ececb05aa62a4eba581ecb"
server_key_iv: "734606bd8f60845c1fa8165c1edd4adc"
server_key_version: "encryptionkey@0"
I20260811 17:13:58.298625 23696 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 17:13:58.301998 23713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:58.303627 23696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:13:58.303839 23696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "01a1ecf48bad4bb3ac29277ac50264c2"
format_stamp: "Formatted at 2026-08-11 17:13:58 on dist-test-slave-fd7l"
server_key: "aeb2c437a5ececb05aa62a4eba581ecb"
server_key_iv: "734606bd8f60845c1fa8165c1edd4adc"
server_key_version: "encryptionkey@0"
I20260811 17:13:58.304132 23696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:13:58.329210 23696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:13:58.333477 23696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:13:58.333910 23696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:13:58.344587 23696 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:35417
I20260811 17:13:58.344615 23783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:35417 every 8 connection(s)
I20260811 17:13:58.345687 23696 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:13:58.346204 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 23696
I20260811 17:13:58.346331 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:13:58.346681 21722 external_mini_cluster.cc:1428] Setting key 8498ee1d8fc6c69a708c0064907234e1
I20260811 17:13:58.349411 23786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:58.355708 23786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2: Bootstrap starting.
Aug 11 17:13:58 dist-test-slave-fd7l krb5kdc[23682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:13:58.358997 23786 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:58.360639 23786 log.cc:824] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2: Log is configured to *not* fsync() on all Append() calls
I20260811 17:13:58.363327 23786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2: No bootstrap required, opened a new log
I20260811 17:13:58.363639 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:58.348249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34374 (local address 127.21.54.190:35417)
0811 17:13:58.348813 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:58.348829 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:58.348870 (+ 41us) server_negotiation.cc:407] Connection header received
0811 17:13:58.349776 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:58.349798 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:58.350181 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:58.350550 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:58.351457 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:58.352483 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:58.353482 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:58.353977 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:58.354174 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:58.357280 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:58.357286 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:58.357301 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:58.357347 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:58.359553 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:58.360230 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:58.360236 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:58.360253 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:58.360400 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:58.360870 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:58.360877 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:58.360880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:58.361533 (+ 653us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:58.361816 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:58.362283 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 17:13:58.362719 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":184,"threads_started":1}
I20260811 17:13:58.366972 23786 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a1ecf48bad4bb3ac29277ac50264c2" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35417 } }
I20260811 17:13:58.367246 23786 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:58.367286 23786 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a1ecf48bad4bb3ac29277ac50264c2, State: Initialized, Role: FOLLOWER
I20260811 17:13:58.367743 23786 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a1ecf48bad4bb3ac29277ac50264c2" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35417 } }
I20260811 17:13:58.367878 23786 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:13:58.367969 23786 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:13:58.368103 23786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:58.369256 23786 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a1ecf48bad4bb3ac29277ac50264c2" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35417 } }
I20260811 17:13:58.369645 23786 leader_election.cc:302] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01a1ecf48bad4bb3ac29277ac50264c2; no voters:
I20260811 17:13:58.370023 23786 leader_election.cc:288] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:13:58.370246 23792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:13:58.370523 23792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2 [term 1 LEADER]: Becoming Leader. State: Replica: 01a1ecf48bad4bb3ac29277ac50264c2, State: Running, Role: LEADER
I20260811 17:13:58.370939 23792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a1ecf48bad4bb3ac29277ac50264c2" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35417 } }
I20260811 17:13:58.371629 23786 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:13:58.373188 23793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01a1ecf48bad4bb3ac29277ac50264c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a1ecf48bad4bb3ac29277ac50264c2" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35417 } } }
I20260811 17:13:58.373188 23794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01a1ecf48bad4bb3ac29277ac50264c2. Latest consensus state: current_term: 1 leader_uuid: "01a1ecf48bad4bb3ac29277ac50264c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a1ecf48bad4bb3ac29277ac50264c2" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35417 } } }
I20260811 17:13:58.373347 23793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2 [sys.catalog]: This master's current role is: LEADER
I20260811 17:13:58.373373 23794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2 [sys.catalog]: This master's current role is: LEADER
I20260811 17:13:58.373790 23809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:13:58.376930 23809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:13:58.383055 23809 catalog_manager.cc:1347] Generated new cluster ID: 1ba57277b0cc41858c5c259e43975244
I20260811 17:13:58.383147 23809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:13:58.395205 23809 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:13:58.396281 23809 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:13:58.404548 23809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2: Generated new TSK 0
I20260811 17:13:58.405432 23809 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
I20260811 17:13:58.442166 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:35417
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:13:58.572566 23820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:13:58.572958 23820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:13:58.573046 23820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:13:58.577440 23820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:13:58.577515 23820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:13:58.577558 23820 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 17:13:58.577574 23820 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 17:13:58.577651 23820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:13:58.583453 23820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.54.190:35417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:13:58.584753 23820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:13:58.585901 23820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:13:58.596596 23827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:58.596670 23825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:58.596613 23829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:58.596714 23820 server_base.cc:1034] running on GCE node
I20260811 17:13:58.597366 23820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:13:58.598016 23820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:13:58.599187 23820 hybrid_clock.cc:648] HybridClock initialized: now 1786468438599136 us; error 63 us; skew 500 ppm
Aug 11 17:13:58 dist-test-slave-fd7l krb5kdc[23682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468438, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:13:58.602339 23820 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:13:58.603318 23820 webserver.cc:460] Webserver started at http://127.21.54.129:35495/ using document root <none> and password file <none>
I20260811 17:13:58.603874 23820 fs_manager.cc:359] Metadata directory not provided
I20260811 17:13:58.603933 23820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:13:58.604117 23820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:13:58.606330 23820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "e366c43c964c44b1a96922d9d2147b45"
format_stamp: "Formatted at 2026-08-11 17:13:58 on dist-test-slave-fd7l"
server_key: "7f062faf3114dbb3411365b494555b69"
server_key_iv: "d822d50b3708ced2e76eb00c4020ed69"
server_key_version: "encryptionkey@0"
I20260811 17:13:58.606884 23820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "e366c43c964c44b1a96922d9d2147b45"
format_stamp: "Formatted at 2026-08-11 17:13:58 on dist-test-slave-fd7l"
server_key: "7f062faf3114dbb3411365b494555b69"
server_key_iv: "d822d50b3708ced2e76eb00c4020ed69"
server_key_version: "encryptionkey@0"
I20260811 17:13:58.611099 23820 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 17:13:58.614056 23836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:58.615770 23820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:13:58.615937 23820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "e366c43c964c44b1a96922d9d2147b45"
format_stamp: "Formatted at 2026-08-11 17:13:58 on dist-test-slave-fd7l"
server_key: "7f062faf3114dbb3411365b494555b69"
server_key_iv: "d822d50b3708ced2e76eb00c4020ed69"
server_key_version: "encryptionkey@0"
I20260811 17:13:58.616096 23820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:13:58.628765 23820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:13:58.632495 23820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:13:58.632864 23820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:13:58.635167 23820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:13:58.635267 23820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:58.635358 23820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:13:58.635411 23820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 17:13:58 dist-test-slave-fd7l krb5kdc[23682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468438, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:13:58.647809 23820 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:37193
I20260811 17:13:58.648190 23971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:37193 every 8 connection(s)
I20260811 17:13:58.649232 23820 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:13:58.650656 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 23820
I20260811 17:13:58.650769 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:13:58.651098 21722 external_mini_cluster.cc:1428] Setting key 552c05851b3ef1996b394f9ebe7f7143
I20260811 17:13:58.652223 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:58.635887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:55535 (local address 127.21.54.190:35417)
0811 17:13:58.636073 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:58.636079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:58.637603 (+ 1524us) server_negotiation.cc:407] Connection header received
0811 17:13:58.638925 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:58.638930 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:58.639001 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:58.639129 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:58.641121 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:58.642079 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:58.643409 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:58.643689 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:58.643817 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:58.647349 (+ 3532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:58.647356 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:58.647361 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:58.647408 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:58.649467 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:58.650302 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:58.650306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:58.650307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:58.650428 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:58.650866 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:58.650870 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:58.650872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:58.651118 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:58.651248 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:58.651899 (+ 651us) server_negotiation.cc:299] Negotiation successful
0811 17:13:58.652060 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 17:13:58.653295 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:58.636879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35417 (local address 127.21.54.129:55535)
0811 17:13:58.637399 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:58.637457 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:58.638637 (+ 1180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:58.639382 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:58.639392 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:58.639998 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:58.640901 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:58.640926 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:58.642262 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:58.642268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:58.643252 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:58.643263 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:58.643879 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:58.643884 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:58.644085 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:58.644892 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:58.644914 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:58.647089 (+ 2175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:58.649627 (+ 2538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:58.649632 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:58.649649 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:58.650130 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:58.650546 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:58.650550 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:58.650552 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:58.650751 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:58.651247 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:58.651254 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:58.651558 (+ 304us) client_negotiation.cc:769] Sending connection context
0811 17:13:58.651779 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 17:13:58.652130 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":146,"threads_started":1}
I20260811 17:13:58.655169 23972 heartbeater.cc:347] Connected to a master server at 127.21.54.190:35417
I20260811 17:13:58.655361 23972 heartbeater.cc:464] Registering TS with master...
I20260811 17:13:58.656062 23972 heartbeater.cc:511] Master 127.21.54.190:35417 requested a full tablet report, sending...
I20260811 17:13:58.658169 23731 ts_manager.cc:194] Registered new tserver with Master: e366c43c964c44b1a96922d9d2147b45 (127.21.54.129:37193)
I20260811 17:13:58.660211 23731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:55535
WARNING: no policy specified for kudu/127.21.54.130@KRBTEST.COM; defaulting to no policy
I20260811 17:13:58.667186 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:58.658013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:54707 (local address 127.21.54.190:35417)
0811 17:13:58.658154 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:58.658162 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:58.658179 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:13:58.658222 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:58.658225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:58.658298 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:58.658380 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:58.659049 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:58.660018 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:58.661315 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:58.661770 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:58.661910 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:58.663462 (+ 1552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:58.663467 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:58.663473 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:58.663508 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:58.665313 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:58.665850 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:58.665853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:58.665854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:58.665943 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:58.666299 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:58.666302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:58.666303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:58.666495 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:58.666647 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:58.666909 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 17:13:58.667031 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":330,"server-negotiator.queue_time_us":57}
I20260811 17:13:58.667219 23981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:58.657657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35417 (local address 127.21.54.129:54707)
0811 17:13:58.657975 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:58.657992 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:58.658103 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:58.658521 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:58.658523 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:58.658732 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:58.658846 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:58.658852 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:58.660191 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:58.660196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:58.661098 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:58.661110 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:58.661948 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:58.661952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:58.662045 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:58.662780 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:58.662796 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:58.663276 (+ 480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:58.665451 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:58.665454 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:58.665455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:58.665739 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:58.666092 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:58.666094 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:58.666096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:58.666192 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:58.666645 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:58.666647 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:58.666766 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 17:13:58.666890 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 17:13:58.667060 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":124,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
I20260811 17:13:58.689558 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:35417
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:13:58.817893 23983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:13:58.818234 23983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:13:58.818323 23983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:13:58.822743 23983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:13:58.822854 23983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:13:58.822934 23983 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 17:13:58.822978 23983 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 17:13:58.823055 23983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:13:58.828912 23983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.54.190:35417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:13:58.830209 23983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:13:58.831354 23983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:13:58.842020 23992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:58.842036 23990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:58.842026 23989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:58.842523 23983 server_base.cc:1034] running on GCE node
I20260811 17:13:58.843019 23983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:13:58.843787 23983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:13:58.844975 23983 hybrid_clock.cc:648] HybridClock initialized: now 1786468438844939 us; error 49 us; skew 500 ppm
Aug 11 17:13:58 dist-test-slave-fd7l krb5kdc[23682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468438, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:13:58.848364 23983 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:13:58.849587 23983 webserver.cc:460] Webserver started at http://127.21.54.130:44987/ using document root <none> and password file <none>
I20260811 17:13:58.850241 23983 fs_manager.cc:359] Metadata directory not provided
I20260811 17:13:58.850312 23983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:13:58.850555 23983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:13:58.852918 23983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c"
format_stamp: "Formatted at 2026-08-11 17:13:58 on dist-test-slave-fd7l"
server_key: "b97f0ed7e0c9a009453d7697267ee258"
server_key_iv: "d23da660f6bb2635c451fecf87e41666"
server_key_version: "encryptionkey@0"
I20260811 17:13:58.853605 23983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c"
format_stamp: "Formatted at 2026-08-11 17:13:58 on dist-test-slave-fd7l"
server_key: "b97f0ed7e0c9a009453d7697267ee258"
server_key_iv: "d23da660f6bb2635c451fecf87e41666"
server_key_version: "encryptionkey@0"
I20260811 17:13:58.858006 23983 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:13:58.860939 24000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:58.862567 23983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:13:58.862733 23983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c"
format_stamp: "Formatted at 2026-08-11 17:13:58 on dist-test-slave-fd7l"
server_key: "b97f0ed7e0c9a009453d7697267ee258"
server_key_iv: "d23da660f6bb2635c451fecf87e41666"
server_key_version: "encryptionkey@0"
I20260811 17:13:58.862895 23983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:13:58.891206 23983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:13:58.895133 23983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:13:58.895337 23983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:13:58.897303 23983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:13:58.897368 23983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:58.897413 23983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:13:58.897454 23983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:58.909657 23983 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:37589
Aug 11 17:13:58 dist-test-slave-fd7l krb5kdc[23682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468438, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:13:58.910741 23983 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:13:58.911671 24142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:37589 every 8 connection(s)
I20260811 17:13:58.915062 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:58.898285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:35661 (local address 127.21.54.190:35417)
0811 17:13:58.898445 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:58.898450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:58.899723 (+ 1273us) server_negotiation.cc:407] Connection header received
0811 17:13:58.901124 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:58.901128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:58.901201 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:58.901310 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:58.903279 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:58.904274 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:58.906452 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:58.906744 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:58.906848 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:58.910638 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:58.910642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:58.910645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:58.910685 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:58.912475 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:58.913300 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:58.913303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:58.913304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:58.913404 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:58.913849 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:58.913852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:58.913854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:58.914089 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:58.914193 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:58.914788 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 17:13:58.914931 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 17:13:58.915962 24010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:58.898825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35417 (local address 127.21.54.130:35661)
0811 17:13:58.899533 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:58.899586 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:58.900852 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:58.901575 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:58.901588 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:58.902198 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:58.903050 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:58.903077 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:58.904489 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:58.904495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:58.906300 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:58.906312 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:58.906942 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:58.906947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:58.907215 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:58.908130 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:58.908149 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:58.910407 (+ 2258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:58.912834 (+ 2427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:58.912840 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:58.912853 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:58.913156 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:58.913502 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:58.913504 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:58.913506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:58.913706 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:58.914209 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:58.914215 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:58.914479 (+ 264us) client_negotiation.cc:769] Sending connection context
0811 17:13:58.914686 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 17:13:58.914982 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":150,"threads_started":1}
I20260811 17:13:58.917524 24143 heartbeater.cc:347] Connected to a master server at 127.21.54.190:35417
I20260811 17:13:58.917722 24143 heartbeater.cc:464] Registering TS with master...
I20260811 17:13:58.918356 24143 heartbeater.cc:511] Master 127.21.54.190:35417 requested a full tablet report, sending...
I20260811 17:13:58.918828 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 23983
I20260811 17:13:58.918949 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:13:58.919270 21722 external_mini_cluster.cc:1428] Setting key 935524fdcae38a236f175cbd0c54c872
I20260811 17:13:58.919540 23731 ts_manager.cc:194] Registered new tserver with Master: d86f2eaa9a0f47d2ba9fb3eed5e07c4c (127.21.54.130:37589)
I20260811 17:13:58.920356 23731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:35661
I20260811 17:13:58.928835 24152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:58.920004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35417 (local address 127.21.54.130:46687)
0811 17:13:58.920630 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:58.920648 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:58.920791 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:58.921099 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:58.921102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:58.921343 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:58.921468 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:58.921474 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:58.922256 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:58.922259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:58.922818 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:58.922826 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:58.923593 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:58.923595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:58.923675 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:58.924410 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:58.924425 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:58.925001 (+ 576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:58.926812 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:58.926817 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:58.926823 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:58.927112 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:58.927535 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:58.927538 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:58.927539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:58.927622 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:58.928239 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:58.928242 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:58.928343 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 17:13:58.928444 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 17:13:58.928677 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":96,"threads_started":1}
I20260811 17:13:58.928994 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:58.919991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:46687 (local address 127.21.54.190:35417)
0811 17:13:58.920146 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:58.920151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:58.920712 (+ 561us) server_negotiation.cc:407] Connection header received
0811 17:13:58.920912 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:58.920915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:58.920966 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:58.921056 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:58.921585 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:58.922102 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:58.922940 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:58.923355 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:58.923482 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:58.925150 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:58.925153 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:58.925155 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:58.925181 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:58.926695 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:58.927226 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:58.927229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:58.927230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:58.927350 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:58.927729 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:58.927735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:58.927740 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:58.928108 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:58.928230 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:58.928591 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 17:13:58.928758 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
WARNING: no policy specified for kudu/127.21.54.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
I20260811 17:13:58.957886 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:35417
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:13:59.090157 24156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:13:59.090528 24156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:13:59.090624 24156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:13:59.095299 24156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:13:59.095436 24156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:13:59.095530 24156 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 17:13:59.095551 24156 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 17:13:59.095590 24156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:13:59.101794 24156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.54.190:35417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:13:59.103188 24156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:13:59.104312 24156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:13:59.115579 24161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:59.115535 24162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:13:59.115684 24164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:13:59.115829 24156 server_base.cc:1034] running on GCE node
I20260811 17:13:59.116575 24156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:13:59.117408 24156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:13:59.118637 24156 hybrid_clock.cc:648] HybridClock initialized: now 1786468439118605 us; error 46 us; skew 500 ppm
Aug 11 17:13:59 dist-test-slave-fd7l krb5kdc[23682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468439, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:13:59.122390 24156 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:13:59.123608 24156 webserver.cc:460] Webserver started at http://127.21.54.131:40657/ using document root <none> and password file <none>
I20260811 17:13:59.124248 24156 fs_manager.cc:359] Metadata directory not provided
I20260811 17:13:59.124347 24156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:13:59.124670 24156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:13:59.127033 24156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "90ec5087c92d494395b2779179fb769a"
format_stamp: "Formatted at 2026-08-11 17:13:59 on dist-test-slave-fd7l"
server_key: "01863e2e9ee243431659d54b74ad72fc"
server_key_iv: "b7a7446ca7548e9d2fcc56b198906843"
server_key_version: "encryptionkey@0"
I20260811 17:13:59.127678 24156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "90ec5087c92d494395b2779179fb769a"
format_stamp: "Formatted at 2026-08-11 17:13:59 on dist-test-slave-fd7l"
server_key: "01863e2e9ee243431659d54b74ad72fc"
server_key_iv: "b7a7446ca7548e9d2fcc56b198906843"
server_key_version: "encryptionkey@0"
I20260811 17:13:59.131999 24156 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:13:59.135080 24171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:59.136770 24156 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:13:59.136943 24156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "90ec5087c92d494395b2779179fb769a"
format_stamp: "Formatted at 2026-08-11 17:13:59 on dist-test-slave-fd7l"
server_key: "01863e2e9ee243431659d54b74ad72fc"
server_key_iv: "b7a7446ca7548e9d2fcc56b198906843"
server_key_version: "encryptionkey@0"
I20260811 17:13:59.137091 24156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:13:59.162312 24156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:13:59.166260 24156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:13:59.166530 24156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:13:59.168603 24156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:13:59.168687 24156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:13:59.168772 24156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:13:59.168820 24156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 17:13:59 dist-test-slave-fd7l krb5kdc[23682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468439, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:13:59.182021 24156 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:46791
I20260811 17:13:59.182385 24306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:46791 every 8 connection(s)
I20260811 17:13:59.183297 24156 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:13:59.185719 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.169686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:58803 (local address 127.21.54.190:35417)
0811 17:13:59.169843 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:59.169847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:59.171019 (+ 1172us) server_negotiation.cc:407] Connection header received
0811 17:13:59.172485 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:59.172492 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:59.172600 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:59.172710 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:59.174407 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.174999 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.176382 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.176836 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.176975 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.181334 (+ 4359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:59.181340 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:59.181348 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:59.181381 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:59.183088 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:59.183768 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:59.183772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:59.183774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:59.183890 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:59.184365 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:59.184368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:59.184369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:59.184707 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:59.184799 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:59.185418 (+ 619us) server_negotiation.cc:299] Negotiation successful
0811 17:13:59.185587 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 17:13:59.186697 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.170250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35417 (local address 127.21.54.131:58803)
0811 17:13:59.170825 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:59.170873 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:59.172092 (+ 1219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:59.172952 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:59.172964 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:59.173494 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:59.174223 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.174248 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.175148 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.175152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.176236 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.176247 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.178219 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.178224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.178469 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.179258 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:59.179277 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:59.181108 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:59.183239 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:59.183246 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:59.183264 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:59.183648 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:59.184021 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:59.184028 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:59.184032 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:59.184234 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:59.184841 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:59.184847 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:59.185143 (+ 296us) client_negotiation.cc:769] Sending connection context
0811 17:13:59.185337 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 17:13:59.185654 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":161,"threads_started":1}
I20260811 17:13:59.187341 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 24156
I20260811 17:13:59.187507 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:13:59.187882 21722 external_mini_cluster.cc:1428] Setting key 2bac1404b4c869693c73ff615e8758d6
I20260811 17:13:59.188252 24307 heartbeater.cc:347] Connected to a master server at 127.21.54.190:35417
I20260811 17:13:59.188503 24307 heartbeater.cc:464] Registering TS with master...
I20260811 17:13:59.189256 24307 heartbeater.cc:511] Master 127.21.54.190:35417 requested a full tablet report, sending...
I20260811 17:13:59.190465 23732 ts_manager.cc:194] Registered new tserver with Master: 90ec5087c92d494395b2779179fb769a (127.21.54.131:46791)
I20260811 17:13:59.191180 23732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:58803
I20260811 17:13:59.198450 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.190835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:42313 (local address 127.21.54.190:35417)
0811 17:13:59.191004 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:59.191010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:59.191313 (+ 303us) server_negotiation.cc:407] Connection header received
0811 17:13:59.191455 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:59.191458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:59.191509 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:59.191608 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:59.192086 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.192684 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.193512 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.193753 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.193866 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.194977 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:59.194981 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:59.194983 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:59.195013 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:59.196608 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:59.197086 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:59.197090 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:59.197092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:59.197189 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:59.197487 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:59.197490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:59.197492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:59.197743 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:59.197863 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:59.198165 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 17:13:59.198285 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 17:13:59.198467 24314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.190788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35417 (local address 127.21.54.131:42313)
0811 17:13:59.191194 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:59.191207 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:59.191336 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:59.191629 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:59.191632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:59.191856 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:59.191958 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.191964 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.192810 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.192814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.193381 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.193389 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.193874 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.193878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.193974 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.194487 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:59.194499 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:59.194833 (+ 334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:59.196739 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:59.196743 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:59.196745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:59.196968 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:59.197287 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:59.197291 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:59.197293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:59.197383 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:59.197864 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:59.197867 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:59.197980 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 17:13:59.198103 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 17:13:59.198275 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":167,"threads_started":1}
I20260811 17:13:59.202409 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 17:13:59 dist-test-slave-fd7l krb5kdc[23682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 17:13:59 dist-test-slave-fd7l krb5kdc[23682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:13:59.239112 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.229514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34390 (local address 127.21.54.190:35417)
0811 17:13:59.229733 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:59.229737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:59.229751 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:13:59.229796 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:59.229798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:59.229840 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:59.229944 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:59.230727 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.231268 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.232263 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.232556 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.232691 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.235515 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:59.235518 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:59.235520 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:59.235555 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:59.237262 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:59.237767 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:59.237771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:59.237775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:59.237866 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:59.238168 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:59.238172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:59.238173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:59.238380 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:59.238526 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:59.238792 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 17:13:59.238924 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 17:13:59.243043 23732 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:13:59.246640 23732 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:13:59.264640 24325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.257728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:37193 (local address 127.0.0.1:60428)
0811 17:13:59.258333 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:59.258396 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:59.258535 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:59.259707 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:59.259716 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:59.259744 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:59.259902 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.259910 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.261501 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.261504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.262926 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.262937 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.263992 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.263996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.264145 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.264201 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 17:13:59.264299 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 17:13:59.264431 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":54,"thread_start_us":96,"threads_started":1}
I20260811 17:13:59.264695 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.258635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34350 (local address 127.21.54.131:46791)
0811 17:13:59.259667 (+ 1032us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:59.259680 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:59.259703 (+ 23us) server_negotiation.cc:407] Connection header received
0811 17:13:59.259775 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:59.259780 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:59.259944 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:59.260095 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:59.260381 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.261271 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.263341 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.263881 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.264015 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.264206 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:59.264392 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 17:13:59.264498 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":835,"thread_start_us":126,"threads_started":1}
I20260811 17:13:59.264990 24327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.258059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:46791 (local address 127.0.0.1:34350)
0811 17:13:59.258801 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:59.258814 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:59.258896 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:59.260160 (+ 1264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:59.260163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:59.260179 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:59.260265 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.260269 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.261399 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.261405 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.263222 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.263230 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.264118 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.264121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.264207 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.264237 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 17:13:59.264342 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 17:13:59.264890 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":459,"thread_start_us":140,"threads_started":1}
I20260811 17:13:59.264808 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.258529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:37589 (local address 127.0.0.1:34312)
0811 17:13:59.259220 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:59.259233 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:59.259327 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:59.259874 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:59.259877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:59.259896 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:59.260012 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.260022 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.261774 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.261779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.262807 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.262816 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.263985 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.263988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.264468 (+ 480us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.264501 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 17:13:59.264607 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 17:13:59.264672 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":350,"thread_start_us":97,"threads_started":1}
I20260811 17:13:59.265482 24326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.257876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60428 (local address 127.21.54.129:37193)
0811 17:13:59.259162 (+ 1286us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:59.259170 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:59.259196 (+ 26us) server_negotiation.cc:407] Connection header received
0811 17:13:59.259287 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:59.259295 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:59.259533 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:59.259696 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:59.260048 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.261353 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.263043 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.263839 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.264628 (+ 789us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.264819 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:59.265206 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 17:13:59.265323 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1145,"thread_start_us":74,"threads_started":1}
I20260811 17:13:59.266194 24330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.258719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34312 (local address 127.21.54.130:37589)
0811 17:13:59.259168 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:59.259180 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:59.259331 (+ 151us) server_negotiation.cc:407] Connection header received
0811 17:13:59.259441 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:59.259445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:59.259590 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:59.259774 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:59.260158 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.261353 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.262939 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.263846 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.263966 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.264177 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:59.265897 (+ 1720us) server_negotiation.cc:299] Negotiation successful
0811 17:13:59.266041 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":93,"threads_started":1}
I20260811 17:13:59.268420 23894 tablet_service.cc:1467] Processing CreateTablet for tablet 9ba0f462bc804442821d9207d407a673 (DEFAULT_TABLE table=test-table [id=e3dab768acd14d7598bec9366550b308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:13:59.268424 24227 tablet_service.cc:1467] Processing CreateTablet for tablet 9ba0f462bc804442821d9207d407a673 (DEFAULT_TABLE table=test-table [id=e3dab768acd14d7598bec9366550b308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:13:59.268631 24061 tablet_service.cc:1467] Processing CreateTablet for tablet 9ba0f462bc804442821d9207d407a673 (DEFAULT_TABLE table=test-table [id=e3dab768acd14d7598bec9366550b308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:13:59.269727 24061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ba0f462bc804442821d9207d407a673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:59.269727 23894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ba0f462bc804442821d9207d407a673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:59.269731 24227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ba0f462bc804442821d9207d407a673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:59.277352 24331 tablet_bootstrap.cc:492] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Bootstrap starting.
I20260811 17:13:59.278112 24333 tablet_bootstrap.cc:492] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a: Bootstrap starting.
I20260811 17:13:59.280766 24331 tablet_bootstrap.cc:654] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:59.280764 24333 tablet_bootstrap.cc:654] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:59.280855 24334 tablet_bootstrap.cc:492] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45: Bootstrap starting.
I20260811 17:13:59.281810 24331 log.cc:824] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Log is configured to *not* fsync() on all Append() calls
I20260811 17:13:59.281810 24333 log.cc:824] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a: Log is configured to *not* fsync() on all Append() calls
I20260811 17:13:59.282995 24334 tablet_bootstrap.cc:654] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:59.283722 24334 log.cc:824] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45: Log is configured to *not* fsync() on all Append() calls
I20260811 17:13:59.284205 24333 tablet_bootstrap.cc:492] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a: No bootstrap required, opened a new log
I20260811 17:13:59.284205 24331 tablet_bootstrap.cc:492] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: No bootstrap required, opened a new log
I20260811 17:13:59.284421 24331 ts_tablet_manager.cc:1397] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260811 17:13:59.284421 24333 ts_tablet_manager.cc:1397] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 17:13:59.285514 24334 tablet_bootstrap.cc:492] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45: No bootstrap required, opened a new log
I20260811 17:13:59.285734 24334 ts_tablet_manager.cc:1397] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 17:13:59.288784 24331 raft_consensus.cc:348] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } }
I20260811 17:13:59.288769 24333 raft_consensus.cc:348] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } }
I20260811 17:13:59.289095 24333 raft_consensus.cc:374] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:59.289095 24331 raft_consensus.cc:374] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:59.289175 24331 raft_consensus.cc:729] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86f2eaa9a0f47d2ba9fb3eed5e07c4c, State: Initialized, Role: FOLLOWER
I20260811 17:13:59.289191 24333 raft_consensus.cc:729] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90ec5087c92d494395b2779179fb769a, State: Initialized, Role: FOLLOWER
I20260811 17:13:59.289718 24333 consensus_queue.cc:260] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } }
I20260811 17:13:59.289718 24331 consensus_queue.cc:260] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } }
I20260811 17:13:59.289970 24334 raft_consensus.cc:348] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } }
I20260811 17:13:59.290319 24334 raft_consensus.cc:374] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:59.290416 24334 raft_consensus.cc:729] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e366c43c964c44b1a96922d9d2147b45, State: Initialized, Role: FOLLOWER
I20260811 17:13:59.290542 24307 heartbeater.cc:503] Master 127.21.54.190:35417 was elected leader, sending a full tablet report...
I20260811 17:13:59.290925 24331 ts_tablet_manager.cc:1428] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260811 17:13:59.290923 24333 ts_tablet_manager.cc:1428] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 17:13:59.290932 24334 consensus_queue.cc:260] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } }
I20260811 17:13:59.291273 24143 heartbeater.cc:503] Master 127.21.54.190:35417 was elected leader, sending a full tablet report...
I20260811 17:13:59.292869 23972 heartbeater.cc:503] Master 127.21.54.190:35417 was elected leader, sending a full tablet report...
I20260811 17:13:59.294627 24334 ts_tablet_manager.cc:1428] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
W20260811 17:13:59.401041 23973 tablet.cc:2369] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:13:59.412768 24145 tablet.cc:2369] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:13:59.434597 24308 tablet.cc:2369] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:13:59.578112 24341 raft_consensus.cc:482] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:13:59.578460 24341 raft_consensus.cc:504] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } }
I20260811 17:13:59.579870 24341 leader_election.cc:288] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90ec5087c92d494395b2779179fb769a (127.21.54.131:46791), d86f2eaa9a0f47d2ba9fb3eed5e07c4c (127.21.54.130:37589)
I20260811 17:13:59.585799 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.580377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:42335 (local address 127.21.54.131:46791)
0811 17:13:59.580584 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:59.580590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:59.580825 (+ 235us) server_negotiation.cc:407] Connection header received
0811 17:13:59.581031 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:59.581035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:59.581104 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:59.581197 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:59.581590 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.582524 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.584307 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.584981 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.585133 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.585193 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:59.585564 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 17:13:59.585657 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 17:13:59.585799 24330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.580971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:42563 (local address 127.21.54.130:37589)
0811 17:13:59.581159 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:59.581167 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:59.581344 (+ 177us) server_negotiation.cc:407] Connection header received
0811 17:13:59.581431 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:59.581434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:59.581474 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:59.581579 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:59.581837 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.582657 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.584307 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.584981 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.585133 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.585204 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:59.585583 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 17:13:59.585660 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 17:13:59.585881 24343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.580821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:37589 (local address 127.21.54.129:42563)
0811 17:13:59.581233 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:59.581246 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:59.581344 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:59.581610 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:59.581612 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:59.581629 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:59.581723 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.581727 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.582773 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.582776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.584131 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.584137 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.585190 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.585194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.585336 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.585430 (+ 94us) client_negotiation.cc:769] Sending connection context
0811 17:13:59.585526 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 17:13:59.585650 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":114,"thread_start_us":101,"threads_started":1}
I20260811 17:13:59.585880 24342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.580204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:46791 (local address 127.21.54.129:42335)
0811 17:13:59.580665 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:59.580682 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:59.580865 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:59.581233 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:59.581238 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:59.581266 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:59.581450 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.581458 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.582665 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.582668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.584121 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.584136 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.585190 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.585194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.585364 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.585430 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 17:13:59.585526 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 17:13:59.585669 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":152,"thread_start_us":165,"threads_started":1}
I20260811 17:13:59.586601 24085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ba0f462bc804442821d9207d407a673" candidate_uuid: "e366c43c964c44b1a96922d9d2147b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" is_pre_election: true
I20260811 17:13:59.586905 24251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ba0f462bc804442821d9207d407a673" candidate_uuid: "e366c43c964c44b1a96922d9d2147b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ec5087c92d494395b2779179fb769a" is_pre_election: true
I20260811 17:13:59.586961 24085 raft_consensus.cc:2454] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e366c43c964c44b1a96922d9d2147b45 in term 0.
I20260811 17:13:59.587209 24251 raft_consensus.cc:2454] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e366c43c964c44b1a96922d9d2147b45 in term 0.
I20260811 17:13:59.587502 23837 leader_election.cc:302] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86f2eaa9a0f47d2ba9fb3eed5e07c4c, e366c43c964c44b1a96922d9d2147b45; no voters:
I20260811 17:13:59.587831 24341 raft_consensus.cc:2781] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:13:59.587942 24341 raft_consensus.cc:482] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:13:59.588008 24341 raft_consensus.cc:3037] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:59.589300 24341 raft_consensus.cc:504] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } }
I20260811 17:13:59.589711 24341 leader_election.cc:288] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [CANDIDATE]: Term 1 election: Requested vote from peers 90ec5087c92d494395b2779179fb769a (127.21.54.131:46791), d86f2eaa9a0f47d2ba9fb3eed5e07c4c (127.21.54.130:37589)
I20260811 17:13:59.590169 24251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ba0f462bc804442821d9207d407a673" candidate_uuid: "e366c43c964c44b1a96922d9d2147b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ec5087c92d494395b2779179fb769a"
I20260811 17:13:59.590231 24085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ba0f462bc804442821d9207d407a673" candidate_uuid: "e366c43c964c44b1a96922d9d2147b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c"
I20260811 17:13:59.590304 24251 raft_consensus.cc:3037] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:59.590400 24085 raft_consensus.cc:3037] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:59.591629 24085 raft_consensus.cc:2454] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e366c43c964c44b1a96922d9d2147b45 in term 1.
I20260811 17:13:59.591735 24251 raft_consensus.cc:2454] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e366c43c964c44b1a96922d9d2147b45 in term 1.
I20260811 17:13:59.592177 23837 leader_election.cc:302] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90ec5087c92d494395b2779179fb769a, e366c43c964c44b1a96922d9d2147b45; no voters:
I20260811 17:13:59.592427 24341 raft_consensus.cc:2781] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:13:59.592773 24341 raft_consensus.cc:686] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [term 1 LEADER]: Becoming Leader. State: Replica: e366c43c964c44b1a96922d9d2147b45, State: Running, Role: LEADER
I20260811 17:13:59.593221 24341 consensus_queue.cc:237] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } }
I20260811 17:13:59.596907 23731 catalog_manager.cc:5495] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 reported cstate change: term changed from 0 to 1, leader changed from <none> to e366c43c964c44b1a96922d9d2147b45 (127.21.54.129). New cstate: current_term: 1 leader_uuid: "e366c43c964c44b1a96922d9d2147b45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } health_report { overall_health: UNKNOWN } } }
Aug 11 17:13:59 dist-test-slave-fd7l krb5kdc[23682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468438, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:13:59.652168 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.642709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34394 (local address 127.21.54.190:35417)
0811 17:13:59.642881 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:59.642886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:59.643080 (+ 194us) server_negotiation.cc:407] Connection header received
0811 17:13:59.643253 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:59.643258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:59.643302 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:59.643385 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:13:59.644216 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.644841 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.645696 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.645914 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.646031 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.648725 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:13:59.648730 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:13:59.648733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:13:59.648764 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:13:59.650299 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:59.650808 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:59.650813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:59.650815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:59.650894 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:13:59.651185 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:13:59.651190 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:13:59.651192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:13:59.651379 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:13:59.651486 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:59.651821 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 17:13:59.652035 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":144,"server-negotiator.queue_time_us":57}
I20260811 17:13:59.652168 24351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.642668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35417 (local address 127.0.0.1:34394)
0811 17:13:59.642965 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:59.642978 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:59.643146 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:59.643407 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:59.643409 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:59.643965 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:13:59.644071 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.644078 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.645007 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.645011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.645571 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.645580 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.646038 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.646042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.646167 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.646867 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:13:59.646881 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:13:59.648425 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:13:59.650429 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:59.650440 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:59.650447 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:59.650696 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:59.650996 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:13:59.650999 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:13:59.651001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:13:59.651090 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:13:59.651507 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:13:59.651515 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:13:59.651635 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 17:13:59.651755 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 17:13:59.651966 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":3,"thread_start_us":110,"threads_started":1}
I20260811 17:13:59.657467 23731 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:34394:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 17:13:59.658457 23731 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:13:59.664199 23894 tablet_service.cc:1467] Processing CreateTablet for tablet b44155a87f644e7fbecc4697f9bbbadf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9428f68317414f9d82ab07442ff715c5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 17:13:59.664551 24227 tablet_service.cc:1467] Processing CreateTablet for tablet b44155a87f644e7fbecc4697f9bbbadf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9428f68317414f9d82ab07442ff715c5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 17:13:59.664465 24061 tablet_service.cc:1467] Processing CreateTablet for tablet b44155a87f644e7fbecc4697f9bbbadf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9428f68317414f9d82ab07442ff715c5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 17:13:59.665100 23894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b44155a87f644e7fbecc4697f9bbbadf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:59.665105 24227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b44155a87f644e7fbecc4697f9bbbadf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:59.665177 24061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b44155a87f644e7fbecc4697f9bbbadf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:59.668031 24331 tablet_bootstrap.cc:492] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Bootstrap starting.
I20260811 17:13:59.668164 24334 tablet_bootstrap.cc:492] T b44155a87f644e7fbecc4697f9bbbadf P e366c43c964c44b1a96922d9d2147b45: Bootstrap starting.
I20260811 17:13:59.669077 24331 tablet_bootstrap.cc:654] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:59.669178 24334 tablet_bootstrap.cc:654] T b44155a87f644e7fbecc4697f9bbbadf P e366c43c964c44b1a96922d9d2147b45: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:59.669809 24333 tablet_bootstrap.cc:492] T b44155a87f644e7fbecc4697f9bbbadf P 90ec5087c92d494395b2779179fb769a: Bootstrap starting.
I20260811 17:13:59.670647 24334 tablet_bootstrap.cc:492] T b44155a87f644e7fbecc4697f9bbbadf P e366c43c964c44b1a96922d9d2147b45: No bootstrap required, opened a new log
I20260811 17:13:59.670747 24334 ts_tablet_manager.cc:1397] T b44155a87f644e7fbecc4697f9bbbadf P e366c43c964c44b1a96922d9d2147b45: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:13:59.670800 24331 tablet_bootstrap.cc:492] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: No bootstrap required, opened a new log
I20260811 17:13:59.670822 24333 tablet_bootstrap.cc:654] T b44155a87f644e7fbecc4697f9bbbadf P 90ec5087c92d494395b2779179fb769a: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:59.670889 24331 ts_tablet_manager.cc:1397] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 17:13:59.671295 24334 raft_consensus.cc:348] T b44155a87f644e7fbecc4697f9bbbadf P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:13:59.671344 24331 raft_consensus.cc:348] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:13:59.671438 24331 raft_consensus.cc:374] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:59.671435 24334 raft_consensus.cc:374] T b44155a87f644e7fbecc4697f9bbbadf P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:59.671470 24334 raft_consensus.cc:729] T b44155a87f644e7fbecc4697f9bbbadf P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e366c43c964c44b1a96922d9d2147b45, State: Initialized, Role: FOLLOWER
I20260811 17:13:59.671470 24331 raft_consensus.cc:729] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86f2eaa9a0f47d2ba9fb3eed5e07c4c, State: Initialized, Role: FOLLOWER
I20260811 17:13:59.671602 24334 consensus_queue.cc:260] T b44155a87f644e7fbecc4697f9bbbadf P e366c43c964c44b1a96922d9d2147b45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:13:59.671602 24331 consensus_queue.cc:260] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:13:59.671877 24331 ts_tablet_manager.cc:1428] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 17:13:59.671877 24334 ts_tablet_manager.cc:1428] T b44155a87f644e7fbecc4697f9bbbadf P e366c43c964c44b1a96922d9d2147b45: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:13:59.672276 24341 tablet_replica.cc:439] T b44155a87f644e7fbecc4697f9bbbadf P e366c43c964c44b1a96922d9d2147b45: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } } }
I20260811 17:13:59.672470 24341 tablet_replica.cc:442] T b44155a87f644e7fbecc4697f9bbbadf P e366c43c964c44b1a96922d9d2147b45: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:13:59.673058 24338 tablet_replica.cc:439] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } } }
I20260811 17:13:59.673231 24338 tablet_replica.cc:442] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:13:59.673561 24333 tablet_bootstrap.cc:492] T b44155a87f644e7fbecc4697f9bbbadf P 90ec5087c92d494395b2779179fb769a: No bootstrap required, opened a new log
I20260811 17:13:59.673689 24333 ts_tablet_manager.cc:1397] T b44155a87f644e7fbecc4697f9bbbadf P 90ec5087c92d494395b2779179fb769a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 17:13:59.674183 24333 raft_consensus.cc:348] T b44155a87f644e7fbecc4697f9bbbadf P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:13:59.674270 24333 raft_consensus.cc:374] T b44155a87f644e7fbecc4697f9bbbadf P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:59.674291 24333 raft_consensus.cc:729] T b44155a87f644e7fbecc4697f9bbbadf P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90ec5087c92d494395b2779179fb769a, State: Initialized, Role: FOLLOWER
I20260811 17:13:59.674418 24333 consensus_queue.cc:260] T b44155a87f644e7fbecc4697f9bbbadf P 90ec5087c92d494395b2779179fb769a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:13:59.674784 24339 tablet_replica.cc:439] T b44155a87f644e7fbecc4697f9bbbadf P 90ec5087c92d494395b2779179fb769a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } } }
I20260811 17:13:59.674960 24339 tablet_replica.cc:442] T b44155a87f644e7fbecc4697f9bbbadf P 90ec5087c92d494395b2779179fb769a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:13:59.675261 24333 ts_tablet_manager.cc:1428] T b44155a87f644e7fbecc4697f9bbbadf P 90ec5087c92d494395b2779179fb769a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 17:13:59.754431 24338 raft_consensus.cc:482] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:13:59.754679 24338 raft_consensus.cc:504] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:13:59.755885 24338 leader_election.cc:288] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e366c43c964c44b1a96922d9d2147b45 (127.21.54.129:37193), 90ec5087c92d494395b2779179fb769a (127.21.54.131:46791)
I20260811 17:13:59.760241 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.756344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:52589 (local address 127.21.54.131:46791)
0811 17:13:59.756485 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:59.756490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:59.756764 (+ 274us) server_negotiation.cc:407] Connection header received
0811 17:13:59.756935 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:59.756939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:59.756988 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:59.757083 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:59.757372 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.757895 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.759141 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.759600 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.759758 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.759821 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:59.760021 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 17:13:59.760097 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 17:13:59.760272 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.756244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:46791 (local address 127.21.54.130:52589)
0811 17:13:59.756658 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:59.756671 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:59.756815 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:59.757093 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:59.757096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:59.757119 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:59.757256 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.757261 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.758013 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.758018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.758896 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.758905 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.759760 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.759764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.759869 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.759893 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 17:13:59.759986 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 17:13:59.760077 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":30,"spinlock_wait_cycles":7936,"thread_start_us":71,"threads_started":1}
I20260811 17:13:59.760288 24326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.756344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:36617 (local address 127.21.54.129:37193)
0811 17:13:59.756485 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:59.756490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:59.756751 (+ 261us) server_negotiation.cc:407] Connection header received
0811 17:13:59.756925 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:59.756929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:59.756987 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:59.757077 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:13:59.757767 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.758274 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.759294 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.759703 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.759836 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.759888 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:59.760088 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 17:13:59.760170 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 17:13:59.760316 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.756244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:37193 (local address 127.21.54.130:36617)
0811 17:13:59.756637 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:59.756651 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:59.756795 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:59.757091 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:59.757094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:59.757126 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:13:59.757635 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.757642 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.758402 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.758407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.759158 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.759167 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.759834 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.759837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.759959 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.759977 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 17:13:59.760068 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 17:13:59.760139 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":6,"spinlock_wait_cycles":4480,"thread_start_us":93,"threads_started":1}
I20260811 17:13:59.760942 24251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b44155a87f644e7fbecc4697f9bbbadf" candidate_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ec5087c92d494395b2779179fb769a" is_pre_election: true
I20260811 17:13:59.761003 23896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b44155a87f644e7fbecc4697f9bbbadf" candidate_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e366c43c964c44b1a96922d9d2147b45" is_pre_election: true
I20260811 17:13:59.761126 24251 raft_consensus.cc:2454] T b44155a87f644e7fbecc4697f9bbbadf P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d86f2eaa9a0f47d2ba9fb3eed5e07c4c in term 0.
I20260811 17:13:59.761264 23896 raft_consensus.cc:2454] T b44155a87f644e7fbecc4697f9bbbadf P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d86f2eaa9a0f47d2ba9fb3eed5e07c4c in term 0.
I20260811 17:13:59.761600 24001 leader_election.cc:302] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90ec5087c92d494395b2779179fb769a, d86f2eaa9a0f47d2ba9fb3eed5e07c4c; no voters:
I20260811 17:13:59.761958 24338 raft_consensus.cc:2781] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:13:59.762034 24338 raft_consensus.cc:482] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:13:59.762064 24338 raft_consensus.cc:3037] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:59.763162 24338 raft_consensus.cc:504] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:13:59.763646 24338 leader_election.cc:288] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [CANDIDATE]: Term 1 election: Requested vote from peers e366c43c964c44b1a96922d9d2147b45 (127.21.54.129:37193), 90ec5087c92d494395b2779179fb769a (127.21.54.131:46791)
I20260811 17:13:59.764074 23896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b44155a87f644e7fbecc4697f9bbbadf" candidate_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e366c43c964c44b1a96922d9d2147b45"
I20260811 17:13:59.764130 24251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b44155a87f644e7fbecc4697f9bbbadf" candidate_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ec5087c92d494395b2779179fb769a"
I20260811 17:13:59.764220 23896 raft_consensus.cc:3037] T b44155a87f644e7fbecc4697f9bbbadf P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:59.764304 24251 raft_consensus.cc:3037] T b44155a87f644e7fbecc4697f9bbbadf P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:13:59.765326 23896 raft_consensus.cc:2454] T b44155a87f644e7fbecc4697f9bbbadf P e366c43c964c44b1a96922d9d2147b45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86f2eaa9a0f47d2ba9fb3eed5e07c4c in term 1.
I20260811 17:13:59.765537 24251 raft_consensus.cc:2454] T b44155a87f644e7fbecc4697f9bbbadf P 90ec5087c92d494395b2779179fb769a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86f2eaa9a0f47d2ba9fb3eed5e07c4c in term 1.
I20260811 17:13:59.765825 24004 leader_election.cc:302] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86f2eaa9a0f47d2ba9fb3eed5e07c4c, e366c43c964c44b1a96922d9d2147b45; no voters:
I20260811 17:13:59.766033 24338 raft_consensus.cc:2781] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:13:59.766237 24338 raft_consensus.cc:686] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 LEADER]: Becoming Leader. State: Replica: d86f2eaa9a0f47d2ba9fb3eed5e07c4c, State: Running, Role: LEADER
I20260811 17:13:59.766640 24338 consensus_queue.cc:237] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:13:59.769222 24358 tablet_replica.cc:439] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: TxnStatusTablet state changed. Reason: New leader d86f2eaa9a0f47d2ba9fb3eed5e07c4c. Latest consensus state: current_term: 1 leader_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } } }
I20260811 17:13:59.769411 24358 tablet_replica.cc:442] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: This TxnStatusTablet replica's current role is: LEADER
I20260811 17:13:59.769850 24360 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 17:13:59.770102 24360 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 17:13:59.770530 23731 catalog_manager.cc:5495] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c reported cstate change: term changed from 0 to 1, leader changed from <none> to d86f2eaa9a0f47d2ba9fb3eed5e07c4c (127.21.54.130). New cstate: current_term: 1 leader_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } health_report { overall_health: UNKNOWN } } }
I20260811 17:13:59.805608 24351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.800864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:37589 (local address 127.0.0.1:34326)
0811 17:13:59.801059 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:59.801084 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:59.801189 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:59.801590 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:59.801593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:59.801602 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:13:59.801715 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.801719 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.802817 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.802820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.803560 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.803571 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.804251 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.804254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.804335 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.804387 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:13:59.805271 (+ 884us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:13:59.805290 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 17:13:59.805361 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 17:13:59.805453 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 17:13:59.805605 24330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.800999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34326 (local address 127.21.54.130:37589)
0811 17:13:59.801142 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:59.801146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:59.801164 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:13:59.801348 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:59.801352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:59.801422 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:59.801541 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:13:59.801832 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.802667 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.803702 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.804093 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.804206 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.804566 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:13:59.805148 (+ 582us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:13:59.805267 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:59.805396 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 17:13:59.805459 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 17:13:59.811780 24251 raft_consensus.cc:1261] T b44155a87f644e7fbecc4697f9bbbadf P 90ec5087c92d494395b2779179fb769a [term 1 FOLLOWER]: Refusing update from remote peer d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:13:59.811777 23896 raft_consensus.cc:1261] T b44155a87f644e7fbecc4697f9bbbadf P e366c43c964c44b1a96922d9d2147b45 [term 1 FOLLOWER]: Refusing update from remote peer d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:13:59.812634 24338 consensus_queue.cc:1035] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:13:59.813189 24338 consensus_queue.cc:1035] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:13:59.816629 24339 tablet_replica.cc:439] T b44155a87f644e7fbecc4697f9bbbadf P 90ec5087c92d494395b2779179fb769a: TxnStatusTablet state changed. Reason: New leader d86f2eaa9a0f47d2ba9fb3eed5e07c4c. Latest consensus state: current_term: 1 leader_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } } }
I20260811 17:13:59.816762 24339 tablet_replica.cc:442] T b44155a87f644e7fbecc4697f9bbbadf P 90ec5087c92d494395b2779179fb769a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:13:59.816704 24341 tablet_replica.cc:439] T b44155a87f644e7fbecc4697f9bbbadf P e366c43c964c44b1a96922d9d2147b45: TxnStatusTablet state changed. Reason: New leader d86f2eaa9a0f47d2ba9fb3eed5e07c4c. Latest consensus state: current_term: 1 leader_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } } }
I20260811 17:13:59.816831 24341 tablet_replica.cc:442] T b44155a87f644e7fbecc4697f9bbbadf P e366c43c964c44b1a96922d9d2147b45: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:13:59.820581 24358 tablet_replica.cc:439] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } } }
I20260811 17:13:59.820715 24358 tablet_replica.cc:442] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: This TxnStatusTablet replica's current role is: LEADER
I20260811 17:13:59.820816 24358 tablet_replica.cc:439] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } } }
I20260811 17:13:59.820873 24358 tablet_replica.cc:442] T b44155a87f644e7fbecc4697f9bbbadf P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: This TxnStatusTablet replica's current role is: LEADER
I20260811 17:13:59.822610 24339 tablet_replica.cc:439] T b44155a87f644e7fbecc4697f9bbbadf P 90ec5087c92d494395b2779179fb769a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } } }
I20260811 17:13:59.822739 24339 tablet_replica.cc:442] T b44155a87f644e7fbecc4697f9bbbadf P 90ec5087c92d494395b2779179fb769a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:13:59.822659 24364 mvcc.cc:205] Tried to move back new op lower bound from 7317374729461534720 to 7317374729287155712. Current Snapshot: MvccSnapshot[applied={T|T < 7317374729461534720}]
I20260811 17:13:59.825591 24341 tablet_replica.cc:439] T b44155a87f644e7fbecc4697f9bbbadf P e366c43c964c44b1a96922d9d2147b45: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } } }
I20260811 17:13:59.825747 24341 tablet_replica.cc:442] T b44155a87f644e7fbecc4697f9bbbadf P e366c43c964c44b1a96922d9d2147b45: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:13:59.831975 24326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.827432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60436 (local address 127.21.54.129:37193)
0811 17:13:59.827631 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:59.827636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:59.827653 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:13:59.827849 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:59.827852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:59.827914 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:59.828023 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:13:59.828316 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.829091 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.830370 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.830641 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.830736 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.831052 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:13:59.831481 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:13:59.831628 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:59.831738 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 17:13:59.831792 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260811 17:13:59.844197 24330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.840103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:35621 (local address 127.21.54.130:37589)
0811 17:13:59.840242 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:59.840247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:59.840404 (+ 157us) server_negotiation.cc:407] Connection header received
0811 17:13:59.840586 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:59.840592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:59.840646 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:59.840714 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:13:59.841081 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.841596 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.842909 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.843142 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.843235 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.843688 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:13:59.843774 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:13:59.843851 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:59.843988 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 17:13:59.844070 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 17:13:59.844237 24375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.839948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:37589 (local address 127.21.54.129:35621)
0811 17:13:59.840308 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:59.840324 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:59.840441 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:59.840747 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:59.840750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:59.840759 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:13:59.840967 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.840975 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.841727 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.841730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.842764 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.842776 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.843284 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.843289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.843390 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.843452 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:13:59.843876 (+ 424us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:13:59.843884 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 17:13:59.843966 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 17:13:59.844053 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":137,"threads_started":1}
I20260811 17:13:59.853153 24251 raft_consensus.cc:1261] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a [term 1 FOLLOWER]: Refusing update from remote peer e366c43c964c44b1a96922d9d2147b45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:13:59.853259 24085 raft_consensus.cc:1261] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 FOLLOWER]: Refusing update from remote peer e366c43c964c44b1a96922d9d2147b45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:13:59.853698 24341 consensus_queue.cc:1035] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:13:59.854003 24344 consensus_queue.cc:1035] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:13:59.863797 24363 mvcc.cc:205] Tried to move back new op lower bound from 7317374729632731136 to 7317374728576843776. Current Snapshot: MvccSnapshot[applied={T|T < 7317374729632731136}]
I20260811 17:13:59.880367 24381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.876243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:37193 (local address 127.21.54.130:47063)
0811 17:13:59.876681 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 17:13:59.876698 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:13:59.876811 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:13:59.877060 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:13:59.877063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:13:59.877072 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:13:59.877203 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.877208 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.878223 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.878226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.879121 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:13:59.879131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.879678 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.879681 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:13:59.879746 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.879794 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:13:59.880117 (+ 323us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:13:59.880122 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 17:13:59.880169 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 17:13:59.880240 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":124,"threads_started":1}
I20260811 17:13:59.880383 24326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:13:59.876655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:47063 (local address 127.21.54.129:37193)
0811 17:13:59.876779 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 17:13:59.876783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:13:59.876796 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:13:59.876905 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:13:59.876908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:13:59.876957 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:13:59.877041 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:13:59.877340 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.878104 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.879244 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:13:59.879557 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:13:59.879671 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:13:59.879931 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:13:59.880029 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:13:59.880104 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 17:13:59.880191 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 17:13:59.880252 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 17:13:59.915333 23731 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:13:59.915580 23731 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:13:59.919081 23731 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2: Sending DeleteTablet for 3 replicas of tablet 9ba0f462bc804442821d9207d407a673
I20260811 17:13:59.920250 23894 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ba0f462bc804442821d9207d407a673 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:13:59 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:60428
I20260811 17:13:59.921358 24227 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ba0f462bc804442821d9207d407a673 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:13:59 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:34350
I20260811 17:13:59.921882 24061 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ba0f462bc804442821d9207d407a673 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:13:59 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:34312
I20260811 17:13:59.922111 23731 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:13:59.922680 23731 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:13:59.927690 24386 tablet_replica.cc:330] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a: stopping tablet replica
I20260811 17:13:59.928079 24386 raft_consensus.cc:2229] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:13:59.928320 24386 raft_consensus.cc:2258] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:13:59.928712 23894 tablet_service.cc:1467] Processing CreateTablet for tablet 1b9aaf0978fa495d9dfefd5d06091f9b (DEFAULT_TABLE table=test-table [id=e75f047dd34f4a1c8a22446edbfe1419]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:13:59.929059 23894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b9aaf0978fa495d9dfefd5d06091f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:59.931852 24334 tablet_bootstrap.cc:492] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45: Bootstrap starting.
I20260811 17:13:59.932962 24334 tablet_bootstrap.cc:654] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:59.933975 24061 tablet_service.cc:1467] Processing CreateTablet for tablet 1b9aaf0978fa495d9dfefd5d06091f9b (DEFAULT_TABLE table=test-table [id=e75f047dd34f4a1c8a22446edbfe1419]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:13:59.934295 24385 tablet_replica.cc:330] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45: stopping tablet replica
I20260811 17:13:59.934072 24387 tablet_replica.cc:330] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: stopping tablet replica
I20260811 17:13:59.934556 24227 tablet_service.cc:1467] Processing CreateTablet for tablet 1b9aaf0978fa495d9dfefd5d06091f9b (DEFAULT_TABLE table=test-table [id=e75f047dd34f4a1c8a22446edbfe1419]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:13:59.934834 24227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b9aaf0978fa495d9dfefd5d06091f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:59.935065 24385 raft_consensus.cc:2229] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:13:59.935267 24387 raft_consensus.cc:2229] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:13:59.935496 24387 raft_consensus.cc:2258] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:13:59.935400 24334 tablet_bootstrap.cc:492] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45: No bootstrap required, opened a new log
I20260811 17:13:59.935973 24334 ts_tablet_manager.cc:1397] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 17:13:59.936028 24385 raft_consensus.cc:2258] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:13:59.936430 24334 raft_consensus.cc:348] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:13:59.936655 24334 raft_consensus.cc:374] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:59.936685 24334 raft_consensus.cc:729] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e366c43c964c44b1a96922d9d2147b45, State: Initialized, Role: FOLLOWER
I20260811 17:13:59.934260 24061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b9aaf0978fa495d9dfefd5d06091f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:13:59.936802 24334 consensus_queue.cc:260] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:13:59.937049 24334 ts_tablet_manager.cc:1428] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 17:13:59.937628 24333 tablet_bootstrap.cc:492] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a: Bootstrap starting.
I20260811 17:13:59.938648 24333 tablet_bootstrap.cc:654] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:59.939369 24331 tablet_bootstrap.cc:492] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Bootstrap starting.
I20260811 17:13:59.940380 24331 tablet_bootstrap.cc:654] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Neither blocks nor log segments found. Creating new log.
I20260811 17:13:59.940455 24386 ts_tablet_manager.cc:1905] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:13:59.941682 24385 ts_tablet_manager.cc:1905] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:13:59.943348 24387 ts_tablet_manager.cc:1905] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:13:59.943531 24386 ts_tablet_manager.cc:1918] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:13:59.943634 24386 log.cc:1196] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/9ba0f462bc804442821d9207d407a673
I20260811 17:13:59.943969 24386 ts_tablet_manager.cc:1939] T 9ba0f462bc804442821d9207d407a673 P 90ec5087c92d494395b2779179fb769a: Deleting consensus metadata
I20260811 17:13:59.944789 24385 ts_tablet_manager.cc:1918] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:13:59.944890 24385 log.cc:1196] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/wal/wals/9ba0f462bc804442821d9207d407a673
I20260811 17:13:59.945195 24385 ts_tablet_manager.cc:1939] T 9ba0f462bc804442821d9207d407a673 P e366c43c964c44b1a96922d9d2147b45: Deleting consensus metadata
I20260811 17:13:59.946377 23717 catalog_manager.cc:4841] TS e366c43c964c44b1a96922d9d2147b45 (127.21.54.129:37193): tablet 9ba0f462bc804442821d9207d407a673 (table test-table [id=e3dab768acd14d7598bec9366550b308]) successfully deleted
I20260811 17:13:59.947723 23714 catalog_manager.cc:4841] TS 90ec5087c92d494395b2779179fb769a (127.21.54.131:46791): tablet 9ba0f462bc804442821d9207d407a673 (table test-table [id=e3dab768acd14d7598bec9366550b308]) successfully deleted
I20260811 17:13:59.948766 24331 tablet_bootstrap.cc:492] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: No bootstrap required, opened a new log
I20260811 17:13:59.948882 24331 ts_tablet_manager.cc:1397] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20260811 17:13:59.949026 24333 tablet_bootstrap.cc:492] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a: No bootstrap required, opened a new log
I20260811 17:13:59.949122 24333 ts_tablet_manager.cc:1397] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20260811 17:13:59.949406 24331 raft_consensus.cc:348] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:13:59.949555 24331 raft_consensus.cc:374] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:59.949599 24331 raft_consensus.cc:729] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86f2eaa9a0f47d2ba9fb3eed5e07c4c, State: Initialized, Role: FOLLOWER
I20260811 17:13:59.949596 24333 raft_consensus.cc:348] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:13:59.949717 24333 raft_consensus.cc:374] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:13:59.949761 24333 raft_consensus.cc:729] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90ec5087c92d494395b2779179fb769a, State: Initialized, Role: FOLLOWER
I20260811 17:13:59.949774 24331 consensus_queue.cc:260] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:13:59.949887 24333 consensus_queue.cc:260] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:13:59.949961 24387 ts_tablet_manager.cc:1918] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:13:59.950105 24331 ts_tablet_manager.cc:1428] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 17:13:59.950147 24333 ts_tablet_manager.cc:1428] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:13:59.950143 24387 log.cc:1196] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/9ba0f462bc804442821d9207d407a673
I20260811 17:13:59.950704 24387 ts_tablet_manager.cc:1939] T 9ba0f462bc804442821d9207d407a673 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Deleting consensus metadata
I20260811 17:13:59.952733 23714 catalog_manager.cc:4841] TS d86f2eaa9a0f47d2ba9fb3eed5e07c4c (127.21.54.130:37589): tablet 9ba0f462bc804442821d9207d407a673 (table test-table [id=e3dab768acd14d7598bec9366550b308]) successfully deleted
I20260811 17:14:00.010500 24339 raft_consensus.cc:482] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:14:00.010681 24339 raft_consensus.cc:504] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:00.011915 24339 leader_election.cc:288] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e366c43c964c44b1a96922d9d2147b45 (127.21.54.129:37193), d86f2eaa9a0f47d2ba9fb3eed5e07c4c (127.21.54.130:37589)
I20260811 17:14:00.016736 24392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:00.012176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:37589 (local address 127.21.54.131:57357)
0811 17:14:00.012772 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:00.012784 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:00.012945 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:00.013238 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:00.013241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:00.013259 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:00.013394 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:00.013399 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:00.014516 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:00.014520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:00.015437 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:00.015449 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:00.016210 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:00.016214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:00.016328 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:00.016355 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 17:14:00.016447 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 17:14:00.016553 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":69,"thread_start_us":149,"threads_started":1}
I20260811 17:14:00.016741 24330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:00.012652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:57357 (local address 127.21.54.130:37589)
0811 17:14:00.012781 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:00.012785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:00.012888 (+ 103us) server_negotiation.cc:407] Connection header received
0811 17:14:00.013073 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:00.013076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:00.013132 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:00.013223 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:00.013530 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:00.014349 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:00.015580 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:00.016047 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:00.016171 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:00.016231 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:00.016483 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 17:14:00.016593 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 17:14:00.017338 24326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:00.012277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:44043 (local address 127.21.54.129:37193)
0811 17:14:00.012497 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:00.012502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:00.012880 (+ 378us) server_negotiation.cc:407] Connection header received
0811 17:14:00.013078 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:00.013081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:00.013130 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:00.013177 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:00.013628 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:00.014358 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:00.015999 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:00.016579 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:00.016699 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:00.016751 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:00.017115 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 17:14:00.017195 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260811 17:14:00.017364 24391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:00.012146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:37193 (local address 127.21.54.131:44043)
0811 17:14:00.012667 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:00.012682 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:00.012958 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:00.013312 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:00.013315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:00.013329 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:00.013458 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:00.013464 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:00.014494 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:00.014498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:00.015843 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:00.015855 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:00.016822 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:00.016828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:00.016955 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:00.016990 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 17:14:00.017075 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 17:14:00.017163 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":2,"thread_start_us":131,"threads_started":1}
I20260811 17:14:00.017839 23896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9aaf0978fa495d9dfefd5d06091f9b" candidate_uuid: "90ec5087c92d494395b2779179fb769a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e366c43c964c44b1a96922d9d2147b45" is_pre_election: true
I20260811 17:14:00.018013 23896 raft_consensus.cc:2454] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90ec5087c92d494395b2779179fb769a in term 0.
I20260811 17:14:00.018255 24085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9aaf0978fa495d9dfefd5d06091f9b" candidate_uuid: "90ec5087c92d494395b2779179fb769a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" is_pre_election: true
I20260811 17:14:00.018414 24085 raft_consensus.cc:2454] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90ec5087c92d494395b2779179fb769a in term 0.
I20260811 17:14:00.018507 24175 leader_election.cc:302] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90ec5087c92d494395b2779179fb769a, e366c43c964c44b1a96922d9d2147b45; no voters:
I20260811 17:14:00.018821 24339 raft_consensus.cc:2781] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:14:00.018914 24339 raft_consensus.cc:482] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:14:00.018965 24339 raft_consensus.cc:3037] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:00.020216 24339 raft_consensus.cc:504] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:00.020761 24339 leader_election.cc:288] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [CANDIDATE]: Term 1 election: Requested vote from peers e366c43c964c44b1a96922d9d2147b45 (127.21.54.129:37193), d86f2eaa9a0f47d2ba9fb3eed5e07c4c (127.21.54.130:37589)
I20260811 17:14:00.021162 24085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9aaf0978fa495d9dfefd5d06091f9b" candidate_uuid: "90ec5087c92d494395b2779179fb769a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c"
I20260811 17:14:00.021304 24085 raft_consensus.cc:3037] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:00.021476 23896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9aaf0978fa495d9dfefd5d06091f9b" candidate_uuid: "90ec5087c92d494395b2779179fb769a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e366c43c964c44b1a96922d9d2147b45"
I20260811 17:14:00.021622 23896 raft_consensus.cc:3037] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:00.022583 23896 raft_consensus.cc:2454] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90ec5087c92d494395b2779179fb769a in term 1.
I20260811 17:14:00.022610 24085 raft_consensus.cc:2454] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90ec5087c92d494395b2779179fb769a in term 1.
I20260811 17:14:00.022905 24175 leader_election.cc:302] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90ec5087c92d494395b2779179fb769a, e366c43c964c44b1a96922d9d2147b45; no voters:
I20260811 17:14:00.023168 24339 raft_consensus.cc:2781] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:00.023452 24339 raft_consensus.cc:686] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [term 1 LEADER]: Becoming Leader. State: Replica: 90ec5087c92d494395b2779179fb769a, State: Running, Role: LEADER
I20260811 17:14:00.023715 24339 consensus_queue.cc:237] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:00.026834 23732 catalog_manager.cc:5495] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a reported cstate change: term changed from 0 to 1, leader changed from <none> to 90ec5087c92d494395b2779179fb769a (127.21.54.131). New cstate: current_term: 1 leader_uuid: "90ec5087c92d494395b2779179fb769a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } health_report { overall_health: HEALTHY } } }
I20260811 17:14:00.057860 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:00.053688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34366 (local address 127.21.54.131:46791)
0811 17:14:00.053826 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:00.053831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:00.053846 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:14:00.054052 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:00.054055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:00.054123 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:00.054242 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:14:00.054570 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:00.055252 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:00.056238 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:00.056473 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:00.056665 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:00.056932 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:14:00.057413 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:14:00.057560 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:00.057624 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 17:14:00.057690 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 17:14:00.071444 24397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:00.066548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:37589 (local address 127.21.54.131:47817)
0811 17:14:00.067500 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:00.067514 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:00.067654 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:00.067966 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:00.067968 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:00.067975 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:14:00.068090 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:00.068095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:00.069184 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:00.069187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:00.069801 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:00.069809 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:00.070438 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:00.070441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:00.070711 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:00.070768 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:14:00.071145 (+ 377us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:14:00.071151 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 17:14:00.071214 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 17:14:00.071289 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":852,"thread_start_us":118,"threads_started":1}
I20260811 17:14:00.071482 24330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:00.066651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:47817 (local address 127.21.54.130:37589)
0811 17:14:00.066814 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:00.066819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:00.067669 (+ 850us) server_negotiation.cc:407] Connection header received
0811 17:14:00.067787 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:00.067791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:00.067855 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:00.067951 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:14:00.068215 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:00.069053 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:00.069940 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:00.070274 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:00.070396 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:00.070942 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:14:00.071040 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:14:00.071127 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:00.071259 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 17:14:00.071324 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 17:14:00.078662 23896 raft_consensus.cc:1261] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45 [term 1 FOLLOWER]: Refusing update from remote peer 90ec5087c92d494395b2779179fb769a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:00.078662 24085 raft_consensus.cc:1261] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 FOLLOWER]: Refusing update from remote peer 90ec5087c92d494395b2779179fb769a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:00.079183 24339 consensus_queue.cc:1035] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:00.079428 24393 consensus_queue.cc:1035] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:00.100625 24381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:00.097244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:46791 (local address 127.21.54.130:55323)
0811 17:14:00.097399 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:00.097416 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:00.097509 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:00.097752 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:00.097755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:00.097764 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:14:00.097869 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:00.097882 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:00.098629 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:00.098633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:00.099379 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:00.099390 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:00.099856 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:00.099860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:00.099955 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:00.099991 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:14:00.100318 (+ 327us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:14:00.100325 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 17:14:00.100386 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 17:14:00.100476 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 17:14:00.100625 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:00.097291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:55323 (local address 127.21.54.131:46791)
0811 17:14:00.097397 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:00.097404 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:00.097460 (+ 56us) server_negotiation.cc:407] Connection header received
0811 17:14:00.097597 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:00.097599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:00.097638 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:00.097728 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:14:00.097990 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:00.098514 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:00.099508 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:00.099743 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:00.099856 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:00.100150 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:14:00.100222 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:14:00.100302 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:00.100396 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 17:14:00.100458 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260811 17:14:00.101383 24364 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 17:14:01.670171 23732 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:01.670333 23732 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:01.673527 23732 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2: Sending DeleteTablet for 3 replicas of tablet 1b9aaf0978fa495d9dfefd5d06091f9b
I20260811 17:14:01.674348 23894 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b9aaf0978fa495d9dfefd5d06091f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:01 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:60428
I20260811 17:14:01.674492 24227 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b9aaf0978fa495d9dfefd5d06091f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:01 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:34350
I20260811 17:14:01.674458 24061 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b9aaf0978fa495d9dfefd5d06091f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:01 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:34312
I20260811 17:14:01.674816 24429 tablet_replica.cc:330] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45: stopping tablet replica
I20260811 17:14:01.674996 24429 raft_consensus.cc:2229] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:01.675158 24429 raft_consensus.cc:2258] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:01.676069 24429 ts_tablet_manager.cc:1905] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:01.678861 24430 tablet_replica.cc:330] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a: stopping tablet replica
I20260811 17:14:01.678882 24429 ts_tablet_manager.cc:1918] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 17:14:01.678977 24429 log.cc:1196] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/wal/wals/1b9aaf0978fa495d9dfefd5d06091f9b
I20260811 17:14:01.679059 24430 raft_consensus.cc:2229] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:14:01.679288 24429 ts_tablet_manager.cc:1939] T 1b9aaf0978fa495d9dfefd5d06091f9b P e366c43c964c44b1a96922d9d2147b45: Deleting consensus metadata
I20260811 17:14:01.679533 24430 raft_consensus.cc:2258] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:01.679872 24431 tablet_replica.cc:330] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: stopping tablet replica
I20260811 17:14:01.680095 24431 raft_consensus.cc:2229] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:01.680202 23717 catalog_manager.cc:4841] TS e366c43c964c44b1a96922d9d2147b45 (127.21.54.129:37193): tablet 1b9aaf0978fa495d9dfefd5d06091f9b (table test-table [id=e75f047dd34f4a1c8a22446edbfe1419]) successfully deleted
I20260811 17:14:01.680296 24431 raft_consensus.cc:2258] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:01.680392 24430 ts_tablet_manager.cc:1905] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:01.680990 24431 ts_tablet_manager.cc:1905] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:01.683542 24431 ts_tablet_manager.cc:1918] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 17:14:01.683650 24431 log.cc:1196] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/1b9aaf0978fa495d9dfefd5d06091f9b
I20260811 17:14:01.683905 24430 ts_tablet_manager.cc:1918] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 17:14:01.683969 24431 ts_tablet_manager.cc:1939] T 1b9aaf0978fa495d9dfefd5d06091f9b P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Deleting consensus metadata
I20260811 17:14:01.684046 24430 log.cc:1196] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/1b9aaf0978fa495d9dfefd5d06091f9b
I20260811 17:14:01.684396 24430 ts_tablet_manager.cc:1939] T 1b9aaf0978fa495d9dfefd5d06091f9b P 90ec5087c92d494395b2779179fb769a: Deleting consensus metadata
I20260811 17:14:01.685163 23714 catalog_manager.cc:4841] TS d86f2eaa9a0f47d2ba9fb3eed5e07c4c (127.21.54.130:37589): tablet 1b9aaf0978fa495d9dfefd5d06091f9b (table test-table [id=e75f047dd34f4a1c8a22446edbfe1419]) successfully deleted
I20260811 17:14:01.685387 23714 catalog_manager.cc:4841] TS 90ec5087c92d494395b2779179fb769a (127.21.54.131:46791): tablet 1b9aaf0978fa495d9dfefd5d06091f9b (table test-table [id=e75f047dd34f4a1c8a22446edbfe1419]) successfully deleted
Aug 11 17:14:01 dist-test-slave-fd7l krb5kdc[23682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468441, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 17:14:01 dist-test-slave-fd7l krb5kdc[23682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468441, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:01.709698 24440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:01.698843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34400 (local address 127.21.54.190:35417)
0811 17:14:01.699182 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:01.699194 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:01.699215 (+ 21us) server_negotiation.cc:407] Connection header received
0811 17:14:01.699407 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:01.699411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:01.699472 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:01.699597 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:01.700409 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:01.701109 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:01.702085 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:01.702369 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:01.702498 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:01.705589 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:01.705592 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:01.705594 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:01.705625 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:01.707452 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:01.708012 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:01.708017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:01.708019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:01.708131 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:01.708468 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:01.708473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:01.708475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:01.708764 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:01.708953 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:01.709265 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 17:14:01.709438 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":108,"threads_started":1}
I20260811 17:14:01.713210 23732 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34400:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:14:01.713766 23732 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:14:01.719749 23894 tablet_service.cc:1467] Processing CreateTablet for tablet 2b7c1a2cb61641a09d3d429c6943a787 (DEFAULT_TABLE table=test-table [id=87b1b1c95af94b4aa7b071a6df456495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:01.719877 24227 tablet_service.cc:1467] Processing CreateTablet for tablet 2b7c1a2cb61641a09d3d429c6943a787 (DEFAULT_TABLE table=test-table [id=87b1b1c95af94b4aa7b071a6df456495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:01.719868 24061 tablet_service.cc:1467] Processing CreateTablet for tablet 2b7c1a2cb61641a09d3d429c6943a787 (DEFAULT_TABLE table=test-table [id=87b1b1c95af94b4aa7b071a6df456495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:01.720158 23894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b7c1a2cb61641a09d3d429c6943a787. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:01.720209 24061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b7c1a2cb61641a09d3d429c6943a787. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:01.720209 24227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b7c1a2cb61641a09d3d429c6943a787. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:01.722510 24441 tablet_bootstrap.cc:492] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45: Bootstrap starting.
I20260811 17:14:01.722808 24445 tablet_bootstrap.cc:492] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Bootstrap starting.
I20260811 17:14:01.723548 24441 tablet_bootstrap.cc:654] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:01.723731 24445 tablet_bootstrap.cc:654] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:01.724370 24442 tablet_bootstrap.cc:492] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a: Bootstrap starting.
I20260811 17:14:01.725009 24441 tablet_bootstrap.cc:492] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45: No bootstrap required, opened a new log
I20260811 17:14:01.725183 24441 ts_tablet_manager.cc:1397] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:14:01.725430 24442 tablet_bootstrap.cc:654] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:01.725497 24445 tablet_bootstrap.cc:492] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: No bootstrap required, opened a new log
I20260811 17:14:01.725656 24445 ts_tablet_manager.cc:1397] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:14:01.725759 24441 raft_consensus.cc:348] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:01.725929 24441 raft_consensus.cc:374] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:01.725996 24441 raft_consensus.cc:729] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e366c43c964c44b1a96922d9d2147b45, State: Initialized, Role: FOLLOWER
I20260811 17:14:01.726169 24441 consensus_queue.cc:260] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:01.726358 24445 raft_consensus.cc:348] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:01.726557 24445 raft_consensus.cc:374] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:01.726634 24441 ts_tablet_manager.cc:1428] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:14:01.726642 24445 raft_consensus.cc:729] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86f2eaa9a0f47d2ba9fb3eed5e07c4c, State: Initialized, Role: FOLLOWER
I20260811 17:14:01.726763 24442 tablet_bootstrap.cc:492] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a: No bootstrap required, opened a new log
I20260811 17:14:01.726846 24442 ts_tablet_manager.cc:1397] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:14:01.726848 24445 consensus_queue.cc:260] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:01.727264 24442 raft_consensus.cc:348] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:01.727370 24442 raft_consensus.cc:374] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:01.727404 24442 raft_consensus.cc:729] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90ec5087c92d494395b2779179fb769a, State: Initialized, Role: FOLLOWER
I20260811 17:14:01.727535 24442 consensus_queue.cc:260] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:01.727803 24442 ts_tablet_manager.cc:1428] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 17:14:01.728606 24445 ts_tablet_manager.cc:1428] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260811 17:14:01.950039 24408 raft_consensus.cc:482] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:14:01.950235 24408 raft_consensus.cc:504] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:01.950655 24408 leader_election.cc:288] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e366c43c964c44b1a96922d9d2147b45 (127.21.54.129:37193), 90ec5087c92d494395b2779179fb769a (127.21.54.131:46791)
I20260811 17:14:01.951265 23896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b7c1a2cb61641a09d3d429c6943a787" candidate_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e366c43c964c44b1a96922d9d2147b45" is_pre_election: true
I20260811 17:14:01.951267 24251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b7c1a2cb61641a09d3d429c6943a787" candidate_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ec5087c92d494395b2779179fb769a" is_pre_election: true
I20260811 17:14:01.951416 23896 raft_consensus.cc:2454] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d86f2eaa9a0f47d2ba9fb3eed5e07c4c in term 0.
I20260811 17:14:01.951464 24251 raft_consensus.cc:2454] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d86f2eaa9a0f47d2ba9fb3eed5e07c4c in term 0.
I20260811 17:14:01.951862 24001 leader_election.cc:302] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90ec5087c92d494395b2779179fb769a, d86f2eaa9a0f47d2ba9fb3eed5e07c4c; no voters:
I20260811 17:14:01.952140 24408 raft_consensus.cc:2781] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:14:01.952216 24408 raft_consensus.cc:482] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:14:01.952250 24408 raft_consensus.cc:3037] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:01.953511 24408 raft_consensus.cc:504] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:01.953961 24408 leader_election.cc:288] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [CANDIDATE]: Term 1 election: Requested vote from peers e366c43c964c44b1a96922d9d2147b45 (127.21.54.129:37193), 90ec5087c92d494395b2779179fb769a (127.21.54.131:46791)
I20260811 17:14:01.954613 24251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b7c1a2cb61641a09d3d429c6943a787" candidate_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ec5087c92d494395b2779179fb769a"
I20260811 17:14:01.954655 23896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b7c1a2cb61641a09d3d429c6943a787" candidate_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e366c43c964c44b1a96922d9d2147b45"
I20260811 17:14:01.954792 24251 raft_consensus.cc:3037] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:01.954794 23896 raft_consensus.cc:3037] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:01.955967 24251 raft_consensus.cc:2454] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86f2eaa9a0f47d2ba9fb3eed5e07c4c in term 1.
I20260811 17:14:01.955976 23896 raft_consensus.cc:2454] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d86f2eaa9a0f47d2ba9fb3eed5e07c4c in term 1.
I20260811 17:14:01.956346 24001 leader_election.cc:302] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 90ec5087c92d494395b2779179fb769a, d86f2eaa9a0f47d2ba9fb3eed5e07c4c, e366c43c964c44b1a96922d9d2147b45; no voters:
I20260811 17:14:01.956624 24408 raft_consensus.cc:2781] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:01.956780 24408 raft_consensus.cc:686] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 LEADER]: Becoming Leader. State: Replica: d86f2eaa9a0f47d2ba9fb3eed5e07c4c, State: Running, Role: LEADER
I20260811 17:14:01.956949 24408 consensus_queue.cc:237] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:01.958946 23731 catalog_manager.cc:5495] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c reported cstate change: term changed from 0 to 1, leader changed from <none> to d86f2eaa9a0f47d2ba9fb3eed5e07c4c (127.21.54.130). New cstate: current_term: 1 leader_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } health_report { overall_health: UNKNOWN } } }
I20260811 17:14:01.976367 24453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:01.973003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34332 (local address 127.21.54.130:37589)
0811 17:14:01.973332 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:01.973339 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:01.973352 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:14:01.973390 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:01.973392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:01.973494 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:01.973571 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:14:01.973898 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:01.974489 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:01.975367 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:01.975639 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:01.975722 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:01.975968 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:14:01.976042 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:14:01.976099 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:01.976185 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 17:14:01.976235 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":127,"threads_started":1}
I20260811 17:14:01.986573 24456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:01.982982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:37589 (local address 127.21.54.130:50425)
0811 17:14:01.983351 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:01.983365 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:01.983579 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:01.983827 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:01.983830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:01.983837 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:14:01.983982 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:01.983989 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:01.984786 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:01.984790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:01.985394 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:01.985403 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:01.985858 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:01.985861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:01.985936 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:01.985969 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:14:01.986255 (+ 286us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:14:01.986261 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 17:14:01.986327 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 17:14:01.986412 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":22,"thread_start_us":154,"threads_started":1}
I20260811 17:14:01.986573 24453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:01.983152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:50425 (local address 127.21.54.130:37589)
0811 17:14:01.983325 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:01.983330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:01.983516 (+ 186us) server_negotiation.cc:407] Connection header received
0811 17:14:01.983680 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:01.983683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:01.983728 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:01.983806 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:14:01.984093 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:01.984674 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:01.985526 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:01.985743 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:01.985856 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:01.986090 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:14:01.986160 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:14:01.986240 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:01.986329 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 17:14:01.986395 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":69}
I20260811 17:14:01.992828 23896 raft_consensus.cc:1261] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45 [term 1 FOLLOWER]: Refusing update from remote peer d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:01.992828 24251 raft_consensus.cc:1261] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a [term 1 FOLLOWER]: Refusing update from remote peer d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:01.993227 24413 consensus_queue.cc:1035] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:01.993402 24376 consensus_queue.cc:1035] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:02.030331 23731 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:02.030499 23731 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:02.032833 23731 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2: Sending DeleteTablet for 3 replicas of tablet 2b7c1a2cb61641a09d3d429c6943a787
I20260811 17:14:02.033495 23894 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b7c1a2cb61641a09d3d429c6943a787 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:02 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:60428
I20260811 17:14:02.033614 24061 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b7c1a2cb61641a09d3d429c6943a787 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:02 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:34312
I20260811 17:14:02.033737 24227 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b7c1a2cb61641a09d3d429c6943a787 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:02 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:34350
I20260811 17:14:02.033809 24431 tablet_replica.cc:330] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: stopping tablet replica
I20260811 17:14:02.033957 24430 tablet_replica.cc:330] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a: stopping tablet replica
I20260811 17:14:02.033965 24429 tablet_replica.cc:330] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45: stopping tablet replica
I20260811 17:14:02.033953 24431 raft_consensus.cc:2229] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:14:02.034094 24429 raft_consensus.cc:2229] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:02.034112 24430 raft_consensus.cc:2229] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:02.034216 24429 raft_consensus.cc:2258] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:02.034231 24430 raft_consensus.cc:2258] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:02.034300 24431 raft_consensus.cc:2258] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:02.034996 24430 ts_tablet_manager.cc:1905] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:02.034996 24431 ts_tablet_manager.cc:1905] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:02.035787 23731 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34400:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:14:02.036404 23731 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:14:02.037796 24430 ts_tablet_manager.cc:1918] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:14:02.037796 24431 ts_tablet_manager.cc:1918] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:14:02.037878 24430 log.cc:1196] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/2b7c1a2cb61641a09d3d429c6943a787
I20260811 17:14:02.037878 24431 log.cc:1196] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/2b7c1a2cb61641a09d3d429c6943a787
I20260811 17:14:02.038154 24431 ts_tablet_manager.cc:1939] T 2b7c1a2cb61641a09d3d429c6943a787 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Deleting consensus metadata
I20260811 17:14:02.038154 24430 ts_tablet_manager.cc:1939] T 2b7c1a2cb61641a09d3d429c6943a787 P 90ec5087c92d494395b2779179fb769a: Deleting consensus metadata
I20260811 17:14:02.038884 23714 catalog_manager.cc:4841] TS d86f2eaa9a0f47d2ba9fb3eed5e07c4c (127.21.54.130:37589): tablet 2b7c1a2cb61641a09d3d429c6943a787 (table test-table [id=87b1b1c95af94b4aa7b071a6df456495]) successfully deleted
I20260811 17:14:02.039253 23714 catalog_manager.cc:4841] TS 90ec5087c92d494395b2779179fb769a (127.21.54.131:46791): tablet 2b7c1a2cb61641a09d3d429c6943a787 (table test-table [id=87b1b1c95af94b4aa7b071a6df456495]) successfully deleted
I20260811 17:14:02.040776 24429 ts_tablet_manager.cc:1905] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:02.041945 24061 tablet_service.cc:1467] Processing CreateTablet for tablet 823680caaa7640dc9b0540c801b5a883 (DEFAULT_TABLE table=test-table [id=c37c0cf447b24cb2bc5d42274d0964d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:02.042214 24061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 823680caaa7640dc9b0540c801b5a883. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:02.043412 24429 ts_tablet_manager.cc:1918] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:14:02.043485 24429 log.cc:1196] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-0/wal/wals/2b7c1a2cb61641a09d3d429c6943a787
I20260811 17:14:02.043736 24429 ts_tablet_manager.cc:1939] T 2b7c1a2cb61641a09d3d429c6943a787 P e366c43c964c44b1a96922d9d2147b45: Deleting consensus metadata
I20260811 17:14:02.044332 24445 tablet_bootstrap.cc:492] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Bootstrap starting.
I20260811 17:14:02.044628 24227 tablet_service.cc:1467] Processing CreateTablet for tablet 823680caaa7640dc9b0540c801b5a883 (DEFAULT_TABLE table=test-table [id=c37c0cf447b24cb2bc5d42274d0964d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:02.044888 24227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 823680caaa7640dc9b0540c801b5a883. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:02.045168 24445 tablet_bootstrap.cc:654] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:02.046139 24445 tablet_bootstrap.cc:492] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: No bootstrap required, opened a new log
I20260811 17:14:02.046201 24445 ts_tablet_manager.cc:1397] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 17:14:02.046542 24445 raft_consensus.cc:348] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:02.046636 24445 raft_consensus.cc:374] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:02.046657 24445 raft_consensus.cc:729] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86f2eaa9a0f47d2ba9fb3eed5e07c4c, State: Initialized, Role: FOLLOWER
I20260811 17:14:02.046790 24445 consensus_queue.cc:260] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:02.047075 24445 ts_tablet_manager.cc:1428] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:14:02.047142 24442 tablet_bootstrap.cc:492] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a: Bootstrap starting.
I20260811 17:14:02.047647 23717 catalog_manager.cc:4841] TS e366c43c964c44b1a96922d9d2147b45 (127.21.54.129:37193): tablet 2b7c1a2cb61641a09d3d429c6943a787 (table test-table [id=87b1b1c95af94b4aa7b071a6df456495]) successfully deleted
I20260811 17:14:02.048064 24442 tablet_bootstrap.cc:654] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:02.048269 23894 tablet_service.cc:1467] Processing CreateTablet for tablet 823680caaa7640dc9b0540c801b5a883 (DEFAULT_TABLE table=test-table [id=c37c0cf447b24cb2bc5d42274d0964d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:02.048543 23894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 823680caaa7640dc9b0540c801b5a883. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:02.050869 24441 tablet_bootstrap.cc:492] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45: Bootstrap starting.
I20260811 17:14:02.051827 24441 tablet_bootstrap.cc:654] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:02.052349 24442 tablet_bootstrap.cc:492] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a: No bootstrap required, opened a new log
I20260811 17:14:02.052479 24442 ts_tablet_manager.cc:1397] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260811 17:14:02.053015 24441 tablet_bootstrap.cc:492] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45: No bootstrap required, opened a new log
I20260811 17:14:02.053099 24441 ts_tablet_manager.cc:1397] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 17:14:02.053009 24442 raft_consensus.cc:348] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:02.053143 24442 raft_consensus.cc:374] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:02.053177 24442 raft_consensus.cc:729] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90ec5087c92d494395b2779179fb769a, State: Initialized, Role: FOLLOWER
I20260811 17:14:02.053309 24442 consensus_queue.cc:260] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:02.053491 24441 raft_consensus.cc:348] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:02.053583 24442 ts_tablet_manager.cc:1428] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 17:14:02.053603 24441 raft_consensus.cc:374] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:02.053637 24441 raft_consensus.cc:729] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e366c43c964c44b1a96922d9d2147b45, State: Initialized, Role: FOLLOWER
I20260811 17:14:02.053755 24441 consensus_queue.cc:260] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:02.054046 24441 ts_tablet_manager.cc:1428] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:14:02.151948 24450 raft_consensus.cc:482] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:14:02.152150 24450 raft_consensus.cc:504] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:02.152657 24450 leader_election.cc:288] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d86f2eaa9a0f47d2ba9fb3eed5e07c4c (127.21.54.130:37589), 90ec5087c92d494395b2779179fb769a (127.21.54.131:46791)
I20260811 17:14:02.153167 24084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "823680caaa7640dc9b0540c801b5a883" candidate_uuid: "e366c43c964c44b1a96922d9d2147b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" is_pre_election: true
I20260811 17:14:02.153316 24084 raft_consensus.cc:2454] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e366c43c964c44b1a96922d9d2147b45 in term 0.
I20260811 17:14:02.153294 24251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "823680caaa7640dc9b0540c801b5a883" candidate_uuid: "e366c43c964c44b1a96922d9d2147b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ec5087c92d494395b2779179fb769a" is_pre_election: true
I20260811 17:14:02.153491 24251 raft_consensus.cc:2454] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e366c43c964c44b1a96922d9d2147b45 in term 0.
I20260811 17:14:02.153800 23837 leader_election.cc:302] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d86f2eaa9a0f47d2ba9fb3eed5e07c4c, e366c43c964c44b1a96922d9d2147b45; no voters:
I20260811 17:14:02.154004 24450 raft_consensus.cc:2781] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:14:02.154088 24450 raft_consensus.cc:482] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:14:02.154147 24450 raft_consensus.cc:3037] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:02.156010 24450 raft_consensus.cc:504] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:02.156589 24450 leader_election.cc:288] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 [CANDIDATE]: Term 1 election: Requested vote from peers d86f2eaa9a0f47d2ba9fb3eed5e07c4c (127.21.54.130:37589), 90ec5087c92d494395b2779179fb769a (127.21.54.131:46791)
I20260811 17:14:02.157083 24084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "823680caaa7640dc9b0540c801b5a883" candidate_uuid: "e366c43c964c44b1a96922d9d2147b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c"
I20260811 17:14:02.157171 24251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "823680caaa7640dc9b0540c801b5a883" candidate_uuid: "e366c43c964c44b1a96922d9d2147b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ec5087c92d494395b2779179fb769a"
I20260811 17:14:02.157243 24084 raft_consensus.cc:3037] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:02.157301 24251 raft_consensus.cc:3037] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:02.158551 24251 raft_consensus.cc:2454] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e366c43c964c44b1a96922d9d2147b45 in term 1.
I20260811 17:14:02.158776 24084 raft_consensus.cc:2454] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e366c43c964c44b1a96922d9d2147b45 in term 1.
I20260811 17:14:02.158928 23837 leader_election.cc:302] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90ec5087c92d494395b2779179fb769a, e366c43c964c44b1a96922d9d2147b45; no voters:
I20260811 17:14:02.159106 24450 raft_consensus.cc:2781] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:02.159322 24450 raft_consensus.cc:686] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 [term 1 LEADER]: Becoming Leader. State: Replica: e366c43c964c44b1a96922d9d2147b45, State: Running, Role: LEADER
I20260811 17:14:02.159540 24450 consensus_queue.cc:237] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } }
I20260811 17:14:02.162038 23732 catalog_manager.cc:5495] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 reported cstate change: term changed from 0 to 1, leader changed from <none> to e366c43c964c44b1a96922d9d2147b45 (127.21.54.129). New cstate: current_term: 1 leader_uuid: "e366c43c964c44b1a96922d9d2147b45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e366c43c964c44b1a96922d9d2147b45" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 } health_report { overall_health: UNKNOWN } } }
I20260811 17:14:02.202701 24467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:02.198146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60442 (local address 127.21.54.129:37193)
0811 17:14:02.198580 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:02.198590 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:02.198604 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:14:02.198680 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:02.198683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:02.198733 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:02.199158 (+ 425us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:14:02.199264 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:02.200085 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:02.201330 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:02.201703 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:02.201806 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:02.202111 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:14:02.202215 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:14:02.202348 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:02.202463 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 17:14:02.202526 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":158,"threads_started":1}
I20260811 17:14:02.211119 24084 raft_consensus.cc:1261] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 FOLLOWER]: Refusing update from remote peer e366c43c964c44b1a96922d9d2147b45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:02.211210 24251 raft_consensus.cc:1261] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a [term 1 FOLLOWER]: Refusing update from remote peer e366c43c964c44b1a96922d9d2147b45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:02.211674 24450 consensus_queue.cc:1035] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90ec5087c92d494395b2779179fb769a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 46791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:02.211855 24464 consensus_queue.cc:1035] T 823680caaa7640dc9b0540c801b5a883 P e366c43c964c44b1a96922d9d2147b45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86f2eaa9a0f47d2ba9fb3eed5e07c4c" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 37589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:02.216817 24410 mvcc.cc:205] Tried to move back new op lower bound from 7317374739292835840 to 7317374739087482880. Current Snapshot: MvccSnapshot[applied={T|T < 7317374739292835840}]
W20260811 17:14:02.230087 24410 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 17:14:03.787122 23732 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:03.787285 23732 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:03.790650 23732 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01a1ecf48bad4bb3ac29277ac50264c2: Sending DeleteTablet for 3 replicas of tablet 823680caaa7640dc9b0540c801b5a883
I20260811 17:14:03.791491 23894 tablet_service.cc:1514] Processing DeleteTablet for tablet 823680caaa7640dc9b0540c801b5a883 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:03 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:60428
I20260811 17:14:03.791710 24227 tablet_service.cc:1514] Processing DeleteTablet for tablet 823680caaa7640dc9b0540c801b5a883 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:03 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:34350
I20260811 17:14:03.791990 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 23820
I20260811 17:14:03.791931 24061 tablet_service.cc:1514] Processing DeleteTablet for tablet 823680caaa7640dc9b0540c801b5a883 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:03 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:34312
I20260811 17:14:03.795787 24499 tablet_replica.cc:330] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a: stopping tablet replica
I20260811 17:14:03.796082 24499 raft_consensus.cc:2229] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:03.796316 24499 raft_consensus.cc:2258] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:03.797415 24500 tablet_replica.cc:330] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: stopping tablet replica
I20260811 17:14:03.797576 24500 raft_consensus.cc:2229] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:03.797592 24499 ts_tablet_manager.cc:1905] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:03.797731 24500 raft_consensus.cc:2258] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:03.798399 24500 ts_tablet_manager.cc:1905] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:03.800868 24499 ts_tablet_manager.cc:1918] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 17:14:03.800972 24499 log.cc:1196] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/823680caaa7640dc9b0540c801b5a883
I20260811 17:14:03.801385 24499 ts_tablet_manager.cc:1939] T 823680caaa7640dc9b0540c801b5a883 P 90ec5087c92d494395b2779179fb769a: Deleting consensus metadata
I20260811 17:14:03.801594 24500 ts_tablet_manager.cc:1918] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 17:14:03.801687 24500 log.cc:1196] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/823680caaa7640dc9b0540c801b5a883
I20260811 17:14:03.801992 24500 ts_tablet_manager.cc:1939] T 823680caaa7640dc9b0540c801b5a883 P d86f2eaa9a0f47d2ba9fb3eed5e07c4c: Deleting consensus metadata
I20260811 17:14:03.802587 23714 catalog_manager.cc:4841] TS 90ec5087c92d494395b2779179fb769a (127.21.54.131:46791): tablet 823680caaa7640dc9b0540c801b5a883 (table test-table [id=c37c0cf447b24cb2bc5d42274d0964d5]) successfully deleted
I20260811 17:14:03.803073 23714 catalog_manager.cc:4841] TS d86f2eaa9a0f47d2ba9fb3eed5e07c4c (127.21.54.130:37589): tablet 823680caaa7640dc9b0540c801b5a883 (table test-table [id=c37c0cf447b24cb2bc5d42274d0964d5]) successfully deleted
W20260811 17:14:03.805205 23717 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.54.129:37193 (error 108)
I20260811 17:14:03.806213 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 23983
I20260811 17:14:03.810096 24503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:03.806056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:37193 (local address 127.0.0.1:58268)
0811 17:14:03.806334 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:03.809920 (+ 3586us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.129:37193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":117,"threads_started":1}
W20260811 17:14:03.810858 23717 catalog_manager.cc:4584] TS e366c43c964c44b1a96922d9d2147b45 (127.21.54.129:37193): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 823680caaa7640dc9b0540c801b5a883: Network error: Client connection negotiation failed: client connection to 127.21.54.129:37193: connect: Connection refused (error 111)
I20260811 17:14:03.818821 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 24156
I20260811 17:14:03.830909 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 23696
2026-08-11T17:14:03Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6998 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:14:03 dist-test-slave-fd7l krb5kdc[24506](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:14:03 dist-test-slave-fd7l krb5kdc[24506](info): set up 2 sockets
Aug 11 17:14:03 dist-test-slave-fd7l krb5kdc[24506](info): commencing operation
krb5kdc: starting...
W20260811 17:14:04.984594 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:14:05 dist-test-slave-fd7l krb5kdc[24506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:14:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:14:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:14:05.141193 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:44219
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:44219
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:05.262497 24520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:05.262880 24520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:05.262960 24520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:05.267313 24520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:05.267436 24520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:05.267474 24520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:05.267511 24520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:05.267539 24520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:14:05.273336 24520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:44219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:44219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:05.274577 24520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:05.275684 24520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:14:05.285471 24520 server_base.cc:1034] running on GCE node
W20260811 17:14:05.285727 24525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:05.285488 24526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:05.285503 24528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:05.286630 24520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:05.287840 24520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:05.289057 24520 hybrid_clock.cc:648] HybridClock initialized: now 1786468445289032 us; error 49 us; skew 500 ppm
Aug 11 17:14:05 dist-test-slave-fd7l krb5kdc[24506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:05.292084 24520 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:14:05.293220 24520 webserver.cc:460] Webserver started at http://127.21.54.190:35819/ using document root <none> and password file <none>
I20260811 17:14:05.293824 24520 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:05.293921 24520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:05.294173 24520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:05.296362 24520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "506ea4db9bb74eb3ab36a6bc27533035"
format_stamp: "Formatted at 2026-08-11 17:14:05 on dist-test-slave-fd7l"
server_key: "0a0d864f405904cb8e248b7ef6171d7e"
server_key_iv: "53be91b5121c2b085082c4bb825454b1"
server_key_version: "encryptionkey@0"
I20260811 17:14:05.297073 24520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "506ea4db9bb74eb3ab36a6bc27533035"
format_stamp: "Formatted at 2026-08-11 17:14:05 on dist-test-slave-fd7l"
server_key: "0a0d864f405904cb8e248b7ef6171d7e"
server_key_iv: "53be91b5121c2b085082c4bb825454b1"
server_key_version: "encryptionkey@0"
I20260811 17:14:05.301239 24520 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:14:05.304021 24537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:05.305625 24520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:05.305804 24520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "506ea4db9bb74eb3ab36a6bc27533035"
format_stamp: "Formatted at 2026-08-11 17:14:05 on dist-test-slave-fd7l"
server_key: "0a0d864f405904cb8e248b7ef6171d7e"
server_key_iv: "53be91b5121c2b085082c4bb825454b1"
server_key_version: "encryptionkey@0"
I20260811 17:14:05.305948 24520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:05.319856 24520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:05.323396 24520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:05.323637 24520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:05.331310 24520 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:44219
I20260811 17:14:05.331338 24599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:44219 every 8 connection(s)
I20260811 17:14:05.332350 24520 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:14:05.335371 24600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:05.339794 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 24520
I20260811 17:14:05.339919 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:14:05.340231 21722 external_mini_cluster.cc:1428] Setting key 2027ac656a732ee1a40ea154dc3d3754
I20260811 17:14:05.342305 24600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035: Bootstrap starting.
I20260811 17:14:05.344772 24600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:05.345614 24600 log.cc:824] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:05.348047 24600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035: No bootstrap required, opened a new log
Aug 11 17:14:05 dist-test-slave-fd7l krb5kdc[24506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:05.352478 24600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506ea4db9bb74eb3ab36a6bc27533035" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44219 } }
I20260811 17:14:05.352823 24600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:05.352914 24600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506ea4db9bb74eb3ab36a6bc27533035, State: Initialized, Role: FOLLOWER
I20260811 17:14:05.353433 24600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506ea4db9bb74eb3ab36a6bc27533035" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44219 } }
I20260811 17:14:05.353597 24600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:14:05.353731 24600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:14:05.353855 24600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:05.355211 24600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506ea4db9bb74eb3ab36a6bc27533035" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44219 } }
I20260811 17:14:05.355603 24600 leader_election.cc:302] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 506ea4db9bb74eb3ab36a6bc27533035; no voters:
I20260811 17:14:05.355968 24600 leader_election.cc:288] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:14:05.356173 24605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:05.356462 24605 raft_consensus.cc:686] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035 [term 1 LEADER]: Becoming Leader. State: Replica: 506ea4db9bb74eb3ab36a6bc27533035, State: Running, Role: LEADER
I20260811 17:14:05.356757 24603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:05.341891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57726 (local address 127.21.54.190:44219)
0811 17:14:05.342424 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:05.342449 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:05.342502 (+ 53us) server_negotiation.cc:407] Connection header received
0811 17:14:05.343209 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:05.343228 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:05.343586 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:05.343950 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:05.344914 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:05.346057 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:05.347293 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:05.347837 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:05.348020 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:05.351097 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:05.351104 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:05.351120 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:05.351150 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:05.353205 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:05.353735 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:05.353744 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:05.353753 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:05.353911 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:05.354209 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:05.354215 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:05.354219 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:05.354777 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:05.354993 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:05.355323 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 17:14:05.355662 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":157,"threads_started":1}
I20260811 17:14:05.356891 24605 consensus_queue.cc:237] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506ea4db9bb74eb3ab36a6bc27533035" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44219 } }
I20260811 17:14:05.357452 24600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:05.359004 24607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 506ea4db9bb74eb3ab36a6bc27533035. Latest consensus state: current_term: 1 leader_uuid: "506ea4db9bb74eb3ab36a6bc27533035" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506ea4db9bb74eb3ab36a6bc27533035" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44219 } } }
I20260811 17:14:05.359154 24607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:05.359798 24606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "506ea4db9bb74eb3ab36a6bc27533035" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506ea4db9bb74eb3ab36a6bc27533035" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44219 } } }
I20260811 17:14:05.359917 24606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:05.360155 24614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:14:05.363449 24614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:14:05.370512 24614 catalog_manager.cc:1347] Generated new cluster ID: fba14ea5c46348a19ecc91aa9b2c5638
I20260811 17:14:05.370617 24614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:14:05.387432 24614 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:14:05.388622 24614 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:14:05.393421 24614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 506ea4db9bb74eb3ab36a6bc27533035: Generated new TSK 0
I20260811 17:14:05.394405 24614 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
I20260811 17:14:05.436852 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44219
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:05.562654 24631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:05.563012 24631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:05.563088 24631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:05.567715 24631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:05.567852 24631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:05.567965 24631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:14:05.573989 24631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:05.575238 24631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:05.576345 24631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:05.587127 24636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:05.587217 24639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:05.587127 24637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:05.587386 24631 server_base.cc:1034] running on GCE node
I20260811 17:14:05.587962 24631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:05.588848 24631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:05.590026 24631 hybrid_clock.cc:648] HybridClock initialized: now 1786468445590023 us; error 52 us; skew 500 ppm
Aug 11 17:14:05 dist-test-slave-fd7l krb5kdc[24506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:05.593392 24631 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:14:05.594556 24631 webserver.cc:460] Webserver started at http://127.21.54.129:35127/ using document root <none> and password file <none>
I20260811 17:14:05.595234 24631 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:05.595340 24631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:05.595610 24631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:05.598055 24631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "9a06fb6b12fd489292148ae731054f3a"
format_stamp: "Formatted at 2026-08-11 17:14:05 on dist-test-slave-fd7l"
server_key: "d9ad73d8a4c13f282e211b461c3e3ea5"
server_key_iv: "eb3509c0dcf65b8f0a9cbe791ebdcdd9"
server_key_version: "encryptionkey@0"
I20260811 17:14:05.598778 24631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "9a06fb6b12fd489292148ae731054f3a"
format_stamp: "Formatted at 2026-08-11 17:14:05 on dist-test-slave-fd7l"
server_key: "d9ad73d8a4c13f282e211b461c3e3ea5"
server_key_iv: "eb3509c0dcf65b8f0a9cbe791ebdcdd9"
server_key_version: "encryptionkey@0"
I20260811 17:14:05.603523 24631 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:14:05.606503 24646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:05.607889 24631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 17:14:05.608078 24631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "9a06fb6b12fd489292148ae731054f3a"
format_stamp: "Formatted at 2026-08-11 17:14:05 on dist-test-slave-fd7l"
server_key: "d9ad73d8a4c13f282e211b461c3e3ea5"
server_key_iv: "eb3509c0dcf65b8f0a9cbe791ebdcdd9"
server_key_version: "encryptionkey@0"
I20260811 17:14:05.608228 24631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:05.624634 24631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:05.628252 24631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:05.628487 24631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:05.629211 24631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:05.630187 24631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:05.630260 24631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:05.630330 24631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:05.630379 24631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:05.640101 24631 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:42447
I20260811 17:14:05.640132 24777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:42447 every 8 connection(s)
I20260811 17:14:05.641182 24631 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:14:05.645995 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 24631
I20260811 17:14:05.646133 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:14:05.646492 21722 external_mini_cluster.cc:1428] Setting key f38759f28eeb1502040b316c3614148f
Aug 11 17:14:05 dist-test-slave-fd7l krb5kdc[24506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:05.658141 24603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:05.643189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:47597 (local address 127.21.54.190:44219)
0811 17:14:05.643379 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:05.643383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:05.644685 (+ 1302us) server_negotiation.cc:407] Connection header received
0811 17:14:05.645605 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:05.645608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:05.645675 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:05.645761 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:05.647561 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:05.648320 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:05.650237 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:05.650500 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:05.650632 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:05.653771 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:05.653774 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:05.653776 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:05.653804 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:05.655458 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:05.656177 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:05.656188 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:05.656189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:05.656275 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:05.656759 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:05.656762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:05.656763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:05.656961 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:05.657045 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:05.657882 (+ 837us) server_negotiation.cc:299] Negotiation successful
0811 17:14:05.658005 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
WARNING: no policy specified for kudu/127.21.54.130@KRBTEST.COM; defaulting to no policy
I20260811 17:14:05.659317 24780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:05.643555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:44219 (local address 127.21.54.129:47597)
0811 17:14:05.644342 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:05.644465 (+ 123us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:05.645327 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:05.646028 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:05.646040 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:05.646620 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:05.647333 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:05.647358 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:05.648569 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:05.648575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:05.650094 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:05.650109 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:05.650681 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:05.650685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:05.650880 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:05.651557 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:05.651574 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:05.653522 (+ 1948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:05.655601 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:05.655610 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:05.655629 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:05.656047 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:05.656376 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:05.656382 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:05.656387 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:05.656641 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:05.657094 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:05.657104 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:05.657548 (+ 444us) client_negotiation.cc:769] Sending connection context
0811 17:14:05.657807 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 17:14:05.658220 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":121,"threads_started":1}
I20260811 17:14:05.660894 24778 heartbeater.cc:347] Connected to a master server at 127.21.54.190:44219
I20260811 17:14:05.661293 24778 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.130@KRBTEST.COM" created.
I20260811 17:14:05.661909 24778 heartbeater.cc:511] Master 127.21.54.190:44219 requested a full tablet report, sending...
I20260811 17:14:05.663901 24555 ts_manager.cc:194] Registered new tserver with Master: 9a06fb6b12fd489292148ae731054f3a (127.21.54.129:42447)
I20260811 17:14:05.665726 24555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:47597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
I20260811 17:14:05.680444 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44219
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:05.804162 24783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:05.804461 24783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:05.804563 24783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:05.809415 24783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:05.809492 24783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:05.809556 24783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:14:05.815348 24783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:05.816601 24783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:05.817720 24783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:05.828356 24790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:05.828464 24789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:05.828542 24783 server_base.cc:1034] running on GCE node
W20260811 17:14:05.828351 24793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:05.829154 24783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:05.829867 24783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:05.831094 24783 hybrid_clock.cc:648] HybridClock initialized: now 1786468445831076 us; error 31 us; skew 500 ppm
Aug 11 17:14:05 dist-test-slave-fd7l krb5kdc[24506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:05.834221 24783 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:14:05.835281 24783 webserver.cc:460] Webserver started at http://127.21.54.130:43677/ using document root <none> and password file <none>
I20260811 17:14:05.835850 24783 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:05.835913 24783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:05.836108 24783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:05.838371 24783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "ba524e232272478d83e0c2166cde5226"
format_stamp: "Formatted at 2026-08-11 17:14:05 on dist-test-slave-fd7l"
server_key: "f00a41d85bbc4d6cd339af36f79086cf"
server_key_iv: "5dfc7727a7578f8b83e764fb3ff25eeb"
server_key_version: "encryptionkey@0"
I20260811 17:14:05.838907 24783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "ba524e232272478d83e0c2166cde5226"
format_stamp: "Formatted at 2026-08-11 17:14:05 on dist-test-slave-fd7l"
server_key: "f00a41d85bbc4d6cd339af36f79086cf"
server_key_iv: "5dfc7727a7578f8b83e764fb3ff25eeb"
server_key_version: "encryptionkey@0"
I20260811 17:14:05.843034 24783 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 17:14:05.845825 24802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:05.847458 24783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:05.847601 24783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "ba524e232272478d83e0c2166cde5226"
format_stamp: "Formatted at 2026-08-11 17:14:05 on dist-test-slave-fd7l"
server_key: "f00a41d85bbc4d6cd339af36f79086cf"
server_key_iv: "5dfc7727a7578f8b83e764fb3ff25eeb"
server_key_version: "encryptionkey@0"
I20260811 17:14:05.847710 24783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:05.864482 24783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:05.868188 24783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:05.868362 24783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:05.869064 24783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:05.869933 24783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:05.869985 24783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:05.870029 24783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:05.870046 24783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:05.879293 24783 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:43989
I20260811 17:14:05.879320 24934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:43989 every 8 connection(s)
I20260811 17:14:05.880303 24783 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:14:05.889537 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 24783
I20260811 17:14:05.889689 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:14:05.890038 21722 external_mini_cluster.cc:1428] Setting key da206bf271966746f913851cddbaace5
Aug 11 17:14:05 dist-test-slave-fd7l krb5kdc[24506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:05.897102 24603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:05.882458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:60615 (local address 127.21.54.190:44219)
0811 17:14:05.882643 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:05.882649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:05.884008 (+ 1359us) server_negotiation.cc:407] Connection header received
0811 17:14:05.885230 (+ 1222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:05.885234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:05.885289 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:05.885383 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:05.886787 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:05.887360 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:05.888951 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:05.889229 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:05.889329 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:05.892816 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:05.892820 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:05.892822 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:05.892847 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:05.894381 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:05.895026 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:05.895029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:05.895031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:05.895124 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:05.895523 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:05.895529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:05.895533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:05.895836 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:05.895952 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:05.896661 (+ 709us) server_negotiation.cc:299] Negotiation successful
0811 17:14:05.896840 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 17:14:05.898084 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:05.882982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:44219 (local address 127.21.54.130:60615)
0811 17:14:05.883785 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:05.883835 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:05.884948 (+ 1113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:05.885622 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:05.885630 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:05.886055 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:05.886600 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:05.886615 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:05.887530 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:05.887533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:05.888777 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:05.888791 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:05.889436 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:05.889441 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:05.889715 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:05.890529 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:05.890552 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:05.892620 (+ 2068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:05.894531 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:05.894539 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:05.894561 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:05.894893 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:05.895224 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:05.895227 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:05.895229 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:05.895404 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:05.896015 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:05.896022 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:05.896325 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 17:14:05.896600 (+ 275us) client_negotiation.cc:240] Negotiation successful
0811 17:14:05.896943 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":127,"threads_started":1}
I20260811 17:14:05.899621 24935 heartbeater.cc:347] Connected to a master server at 127.21.54.190:44219
I20260811 17:14:05.899937 24935 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:05.900697 24935 heartbeater.cc:511] Master 127.21.54.190:44219 requested a full tablet report, sending...
I20260811 17:14:05.902172 24555 ts_manager.cc:194] Registered new tserver with Master: ba524e232272478d83e0c2166cde5226 (127.21.54.130:43989)
WARNING: no policy specified for kudu/127.21.54.131@KRBTEST.COM; defaulting to no policy
I20260811 17:14:05.902891 24555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:60615
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
I20260811 17:14:05.924911 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44219
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:06.046092 24941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:06.046411 24941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:06.046458 24941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:06.050577 24941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:06.050658 24941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:06.050721 24941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:14:06.056895 24941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:06.058233 24941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:06.059473 24941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:06.070942 24947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:06.070942 24946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:06.071099 24941 server_base.cc:1034] running on GCE node
W20260811 17:14:06.071305 24949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:06.071842 24941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:06.072613 24941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:06.073818 24941 hybrid_clock.cc:648] HybridClock initialized: now 1786468446073793 us; error 43 us; skew 500 ppm
Aug 11 17:14:06 dist-test-slave-fd7l krb5kdc[24506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468446, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:06.077065 24941 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:14:06.078109 24941 webserver.cc:460] Webserver started at http://127.21.54.131:43703/ using document root <none> and password file <none>
I20260811 17:14:06.078699 24941 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:06.078763 24941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:06.078956 24941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:06.081243 24941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "aadf40d817e0440d9766f47396419c8c"
format_stamp: "Formatted at 2026-08-11 17:14:06 on dist-test-slave-fd7l"
server_key: "4b4f870e62a8b034038f613db3c302f2"
server_key_iv: "a17736e9a91eb79b3e2be8a977b5feb2"
server_key_version: "encryptionkey@0"
I20260811 17:14:06.081833 24941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "aadf40d817e0440d9766f47396419c8c"
format_stamp: "Formatted at 2026-08-11 17:14:06 on dist-test-slave-fd7l"
server_key: "4b4f870e62a8b034038f613db3c302f2"
server_key_iv: "a17736e9a91eb79b3e2be8a977b5feb2"
server_key_version: "encryptionkey@0"
I20260811 17:14:06.086126 24941 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:14:06.089066 24956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:06.090463 24941 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:06.090653 24941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "aadf40d817e0440d9766f47396419c8c"
format_stamp: "Formatted at 2026-08-11 17:14:06 on dist-test-slave-fd7l"
server_key: "4b4f870e62a8b034038f613db3c302f2"
server_key_iv: "a17736e9a91eb79b3e2be8a977b5feb2"
server_key_version: "encryptionkey@0"
I20260811 17:14:06.090775 24941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:06.110687 24941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:06.114791 24941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:06.115018 24941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:06.115694 24941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:06.116700 24941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:06.116753 24941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:06.116828 24941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:06.116865 24941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:06.126955 24941 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:36483
I20260811 17:14:06.127000 25092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:36483 every 8 connection(s)
I20260811 17:14:06.127955 24941 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:14:06.133589 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 24941
I20260811 17:14:06.133705 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:14:06.133992 21722 external_mini_cluster.cc:1428] Setting key 6165ad2448829a1e29a54b1799e928d8
Aug 11 17:14:06 dist-test-slave-fd7l krb5kdc[24506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468446, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:06.143626 24603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:06.129956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:50091 (local address 127.21.54.190:44219)
0811 17:14:06.130148 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:06.130154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:06.131300 (+ 1146us) server_negotiation.cc:407] Connection header received
0811 17:14:06.132338 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:06.132342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:06.132413 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:06.132565 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:06.133997 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:06.134650 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:06.136109 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:06.136579 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:06.136734 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:06.139441 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:06.139444 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:06.139446 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:06.139482 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:06.141161 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:06.141746 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:06.141749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:06.141750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:06.141843 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:06.142332 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:06.142335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:06.142336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:06.142619 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:06.142714 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:06.143303 (+ 589us) server_negotiation.cc:299] Negotiation successful
0811 17:14:06.143485 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 17:14:06.144647 25096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:06.130372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:44219 (local address 127.21.54.131:50091)
0811 17:14:06.131114 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:06.131159 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:06.132095 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:06.132774 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:06.132782 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:06.133204 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:06.133820 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:06.133839 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:06.134811 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:06.134814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:06.135949 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:06.135960 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:06.136764 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:06.136767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:06.136925 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:06.137625 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:06.137643 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:06.139236 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:06.141289 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:06.141294 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:06.141310 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:06.141594 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:06.141980 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:06.141983 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:06.141984 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:06.142202 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:06.142745 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:06.142752 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:06.143028 (+ 276us) client_negotiation.cc:769] Sending connection context
0811 17:14:06.143210 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 17:14:06.143507 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":107,"threads_started":1}
I20260811 17:14:06.145973 25093 heartbeater.cc:347] Connected to a master server at 127.21.54.190:44219
I20260811 17:14:06.146279 25093 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:06.146916 25093 heartbeater.cc:511] Master 127.21.54.190:44219 requested a full tablet report, sending...
I20260811 17:14:06.148087 24555 ts_manager.cc:194] Registered new tserver with Master: aadf40d817e0440d9766f47396419c8c (127.21.54.131:36483)
I20260811 17:14:06.148690 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:14:06.149047 24555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:50091
W20260811 17:14:06.167667 24603 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:06.166117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57732 (local address 127.21.54.190:44219)
0811 17:14:06.166270 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:06.166274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:06.166375 (+ 101us) server_negotiation.cc:407] Connection header received
0811 17:14:06.166467 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:06.166470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:06.166523 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:06.166609 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:06.167536 (+ 927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57732: BlockingRecv error: recv got EOF from 127.0.0.1:57732 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 17:14:06.169063 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 24631
I20260811 17:14:06.180195 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 24783
I20260811 17:14:06.191017 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 24941
I20260811 17:14:06.202629 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 24520
2026-08-11T17:14:06Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2371 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:14:06 dist-test-slave-fd7l krb5kdc[25106](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:14:06 dist-test-slave-fd7l krb5kdc[25106](info): set up 2 sockets
Aug 11 17:14:06 dist-test-slave-fd7l krb5kdc[25106](info): commencing operation
krb5kdc: starting...
W20260811 17:14:07.382071 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:14:07 dist-test-slave-fd7l krb5kdc[25106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:14:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:14:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:14:07.533533 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:45847
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:46807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:45847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:07.653579 25125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:07.653950 25125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:07.654038 25125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:07.658329 25125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:07.658461 25125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:07.658501 25125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:07.658538 25125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:07.658569 25125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:14:07.664242 25125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:46807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:45847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:45847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:07.665524 25125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:07.666667 25125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:07.676371 25134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:07.676471 25130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:07.676656 25125 server_base.cc:1034] running on GCE node
W20260811 17:14:07.676875 25131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:07.677640 25125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:07.679013 25125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:07.680218 25125 hybrid_clock.cc:648] HybridClock initialized: now 1786468447680190 us; error 52 us; skew 500 ppm
Aug 11 17:14:07 dist-test-slave-fd7l krb5kdc[25106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468447, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:07.683554 25125 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:14:07.684746 25125 webserver.cc:460] Webserver started at http://127.21.54.190:38081/ using document root <none> and password file <none>
I20260811 17:14:07.685386 25125 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:07.685461 25125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:07.685706 25125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:07.688007 25125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "cba1dc6f6e2a457ab8175e0037829ae2"
format_stamp: "Formatted at 2026-08-11 17:14:07 on dist-test-slave-fd7l"
server_key: "08e6c744623674ff8ac2e7c1c93e9767"
server_key_iv: "9a4a78aee1ab81ccbc547a47f8a62746"
server_key_version: "encryptionkey@0"
I20260811 17:14:07.688678 25125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "cba1dc6f6e2a457ab8175e0037829ae2"
format_stamp: "Formatted at 2026-08-11 17:14:07 on dist-test-slave-fd7l"
server_key: "08e6c744623674ff8ac2e7c1c93e9767"
server_key_iv: "9a4a78aee1ab81ccbc547a47f8a62746"
server_key_version: "encryptionkey@0"
I20260811 17:14:07.692870 25125 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:14:07.695654 25141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:07.697188 25125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:07.697361 25125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "cba1dc6f6e2a457ab8175e0037829ae2"
format_stamp: "Formatted at 2026-08-11 17:14:07 on dist-test-slave-fd7l"
server_key: "08e6c744623674ff8ac2e7c1c93e9767"
server_key_iv: "9a4a78aee1ab81ccbc547a47f8a62746"
server_key_version: "encryptionkey@0"
I20260811 17:14:07.697499 25125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:07.721360 25125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:07.725123 25125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:07.725381 25125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:07.733832 25125 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:45847
I20260811 17:14:07.734338 25201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:45847 every 8 connection(s)
I20260811 17:14:07.735008 25125 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:14:07.738372 25202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:07.742506 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 25125
I20260811 17:14:07.742650 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:14:07.742990 21722 external_mini_cluster.cc:1428] Setting key 22cced6e481c5ed5a0e8cdebe314bd4d
I20260811 17:14:07.745564 25202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2: Bootstrap starting.
I20260811 17:14:07.748759 25202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:07.749938 25202 log.cc:824] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:07.753186 25202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2: No bootstrap required, opened a new log
Aug 11 17:14:07 dist-test-slave-fd7l krb5kdc[25106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:07.757318 25202 raft_consensus.cc:348] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba1dc6f6e2a457ab8175e0037829ae2" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45847 } }
I20260811 17:14:07.757601 25202 raft_consensus.cc:374] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:07.757686 25202 raft_consensus.cc:729] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cba1dc6f6e2a457ab8175e0037829ae2, State: Initialized, Role: FOLLOWER
I20260811 17:14:07.758240 25202 consensus_queue.cc:260] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba1dc6f6e2a457ab8175e0037829ae2" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45847 } }
I20260811 17:14:07.758435 25202 raft_consensus.cc:388] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:14:07.758534 25202 raft_consensus.cc:482] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:14:07.758649 25202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:07.760004 25202 raft_consensus.cc:504] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba1dc6f6e2a457ab8175e0037829ae2" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45847 } }
I20260811 17:14:07.760444 25202 leader_election.cc:302] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cba1dc6f6e2a457ab8175e0037829ae2; no voters:
I20260811 17:14:07.760848 25202 leader_election.cc:288] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:14:07.761041 25208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:07.761369 25208 raft_consensus.cc:686] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2 [term 1 LEADER]: Becoming Leader. State: Replica: cba1dc6f6e2a457ab8175e0037829ae2, State: Running, Role: LEADER
I20260811 17:14:07.761473 25206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:07.744537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37076 (local address 127.21.54.190:45847)
0811 17:14:07.744996 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:07.745013 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:07.745052 (+ 39us) server_negotiation.cc:407] Connection header received
0811 17:14:07.745769 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:07.745788 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:07.746141 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:07.746436 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:07.747541 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:07.749097 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:07.750490 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:07.751282 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:07.751448 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:07.754415 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:07.754421 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:07.754440 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:07.754474 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:07.757691 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:07.758227 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:07.758233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:07.758240 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:07.758402 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:07.758822 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:07.758826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:07.758828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:07.759370 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:07.759575 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:07.759975 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 17:14:07.760318 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":141,"threads_started":1}
I20260811 17:14:07.761757 25208 consensus_queue.cc:237] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba1dc6f6e2a457ab8175e0037829ae2" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45847 } }
I20260811 17:14:07.762249 25202 sys_catalog.cc:564] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:07.763182 25208 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cba1dc6f6e2a457ab8175e0037829ae2. Latest consensus state: current_term: 1 leader_uuid: "cba1dc6f6e2a457ab8175e0037829ae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba1dc6f6e2a457ab8175e0037829ae2" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45847 } } }
I20260811 17:14:07.763435 25208 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:07.763731 25209 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cba1dc6f6e2a457ab8175e0037829ae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba1dc6f6e2a457ab8175e0037829ae2" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45847 } } }
I20260811 17:14:07.763836 25209 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:07.764401 25220 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:14:07.765164 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:14:07.767843 25220 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:14:07.774976 25220 catalog_manager.cc:1347] Generated new cluster ID: 2261e1685a224b159418927acc524532
I20260811 17:14:07.775187 25220 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:14:07.798388 25220 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:14:07.799868 25220 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:14:07.800626 25206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:07.788386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37088 (local address 127.21.54.190:45847)
0811 17:14:07.788647 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:07.788653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:07.788911 (+ 258us) server_negotiation.cc:407] Connection header received
0811 17:14:07.789211 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:07.789240 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:07.789296 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:07.789378 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:07.790840 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:07.791774 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:07.793120 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:07.793535 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:07.793650 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:07.799366 (+ 5716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:07.799371 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:07.799374 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:07.800013 (+ 639us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:07.800054 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:07.800076 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:07.800178 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:07.800353 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 17:14:07.800447 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260811 17:14:07.806676 25220 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cba1dc6f6e2a457ab8175e0037829ae2: Generated new TSK 0
I20260811 17:14:07.807475 25220 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:14:07.818575 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 25125
2026-08-11T17:14:07Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1613 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:14:07 dist-test-slave-fd7l krb5kdc[25241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:14:07 dist-test-slave-fd7l krb5kdc[25241](info): set up 2 sockets
Aug 11 17:14:07 dist-test-slave-fd7l krb5kdc[25241](info): commencing operation
krb5kdc: starting...
W20260811 17:14:08.988189 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:14:09 dist-test-slave-fd7l krb5kdc[25241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:14:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:14:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:14:09.136829 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:42029
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:33439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:42029
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:09.260385 25256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:09.260893 25256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:09.260980 25256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:09.265978 25256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:09.266129 25256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:09.266166 25256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:09.266211 25256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:09.266242 25256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:14:09.272370 25256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:33439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:42029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:42029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:09.273743 25256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:09.274878 25256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:09.284981 25262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:09.285149 25256 server_base.cc:1034] running on GCE node
W20260811 17:14:09.284991 25265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:09.284996 25263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:09.286069 25256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:09.287250 25256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:09.288455 25256 hybrid_clock.cc:648] HybridClock initialized: now 1786468449288430 us; error 35 us; skew 500 ppm
Aug 11 17:14:09 dist-test-slave-fd7l krb5kdc[25241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468449, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:09.297207 25256 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:14:09.298458 25256 webserver.cc:460] Webserver started at http://127.21.54.190:36719/ using document root <none> and password file <none>
I20260811 17:14:09.299140 25256 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:09.299252 25256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:09.299516 25256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:09.301914 25256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "19a2683c9c624ecaae0d2399a54081f7"
format_stamp: "Formatted at 2026-08-11 17:14:09 on dist-test-slave-fd7l"
server_key: "6ecf33528960fbb27d4b308673695264"
server_key_iv: "1e9232ef85c94339c69d65521fa068f3"
server_key_version: "encryptionkey@0"
I20260811 17:14:09.302590 25256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "19a2683c9c624ecaae0d2399a54081f7"
format_stamp: "Formatted at 2026-08-11 17:14:09 on dist-test-slave-fd7l"
server_key: "6ecf33528960fbb27d4b308673695264"
server_key_iv: "1e9232ef85c94339c69d65521fa068f3"
server_key_version: "encryptionkey@0"
I20260811 17:14:09.306949 25256 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:14:09.309871 25273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:09.311389 25256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 17:14:09.311581 25256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "19a2683c9c624ecaae0d2399a54081f7"
format_stamp: "Formatted at 2026-08-11 17:14:09 on dist-test-slave-fd7l"
server_key: "6ecf33528960fbb27d4b308673695264"
server_key_iv: "1e9232ef85c94339c69d65521fa068f3"
server_key_version: "encryptionkey@0"
I20260811 17:14:09.311733 25256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:09.324386 25256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:09.328465 25256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:09.328804 25256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:09.337565 25256 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:42029
I20260811 17:14:09.337996 25333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:42029 every 8 connection(s)
I20260811 17:14:09.338757 25256 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:14:09.342008 25334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:09.346321 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 25256
I20260811 17:14:09.346529 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:14:09.346853 21722 external_mini_cluster.cc:1428] Setting key 44e51978a34ad19857611aac5943784e
I20260811 17:14:09.349331 25334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7: Bootstrap starting.
I20260811 17:14:09.352427 25334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:09.353528 25334 log.cc:824] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:09.356038 25334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7: No bootstrap required, opened a new log
Aug 11 17:14:09 dist-test-slave-fd7l krb5kdc[25241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:09.359644 25334 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a2683c9c624ecaae0d2399a54081f7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42029 } }
I20260811 17:14:09.359972 25334 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:09.360114 25334 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19a2683c9c624ecaae0d2399a54081f7, State: Initialized, Role: FOLLOWER
I20260811 17:14:09.360814 25334 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a2683c9c624ecaae0d2399a54081f7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42029 } }
I20260811 17:14:09.360997 25334 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:14:09.361101 25334 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:14:09.361244 25334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:09.362358 25334 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a2683c9c624ecaae0d2399a54081f7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42029 } }
I20260811 17:14:09.362764 25334 leader_election.cc:302] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19a2683c9c624ecaae0d2399a54081f7; no voters:
I20260811 17:14:09.363161 25334 leader_election.cc:288] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:14:09.363251 25340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:09.363767 25340 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7 [term 1 LEADER]: Becoming Leader. State: Replica: 19a2683c9c624ecaae0d2399a54081f7, State: Running, Role: LEADER
I20260811 17:14:09.364144 25340 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a2683c9c624ecaae0d2399a54081f7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42029 } }
I20260811 17:14:09.364614 25338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:09.348666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34974 (local address 127.21.54.190:42029)
0811 17:14:09.349179 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:09.349201 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:09.349246 (+ 45us) server_negotiation.cc:407] Connection header received
0811 17:14:09.350151 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:09.350177 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:09.350621 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:09.351046 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:09.351858 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:09.353484 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:09.354590 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:09.355155 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:09.355379 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:09.358243 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:09.358248 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:09.358264 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:09.358296 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:09.361121 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:09.361656 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:09.361662 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:09.361669 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:09.361806 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:09.362126 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:09.362129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:09.362131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:09.362647 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:09.362847 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:09.363224 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 17:14:09.363588 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":149,"threads_started":1}
I20260811 17:14:09.364720 25334 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:09.365495 25340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19a2683c9c624ecaae0d2399a54081f7. Latest consensus state: current_term: 1 leader_uuid: "19a2683c9c624ecaae0d2399a54081f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a2683c9c624ecaae0d2399a54081f7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42029 } } }
I20260811 17:14:09.365638 25340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:09.365947 25342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19a2683c9c624ecaae0d2399a54081f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a2683c9c624ecaae0d2399a54081f7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42029 } } }
I20260811 17:14:09.366110 25342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:09.366981 25348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:14:09.370012 25348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:14:09.375809 25348 catalog_manager.cc:1347] Generated new cluster ID: ca91ee70272b473cb26d7d03d1a47532
I20260811 17:14:09.375903 25348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:14:09.387923 25348 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:14:09.389047 25348 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:14:09.398206 25348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19a2683c9c624ecaae0d2399a54081f7: Generated new TSK 0
I20260811 17:14:09.399042 25348 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
I20260811 17:14:09.441056 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:42029
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:33439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:09.576102 25364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:09.576417 25364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:09.576463 25364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:09.580739 25364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:09.580863 25364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:09.580966 25364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:14:09.586587 25364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:33439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:42029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:09.587819 25364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:09.588967 25364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:09.599480 25375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:09.599541 25364 server_base.cc:1034] running on GCE node
W20260811 17:14:09.599480 25372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:09.599480 25371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:09.600380 25364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:09.601159 25364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:09.602340 25364 hybrid_clock.cc:648] HybridClock initialized: now 1786468449602324 us; error 27 us; skew 500 ppm
Aug 11 17:14:09 dist-test-slave-fd7l krb5kdc[25241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468449, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:09.605413 25364 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:14:09.606457 25364 webserver.cc:460] Webserver started at http://127.21.54.129:42009/ using document root <none> and password file <none>
I20260811 17:14:09.607100 25364 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:09.607208 25364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:09.607455 25364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:09.609687 25364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "09d7564abdb8490ea06be2e89efd953f"
format_stamp: "Formatted at 2026-08-11 17:14:09 on dist-test-slave-fd7l"
server_key: "6956fadbe57c0375896d1bc74bac003a"
server_key_iv: "08a0bcb3a467873d2790672b09281364"
server_key_version: "encryptionkey@0"
I20260811 17:14:09.610299 25364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "09d7564abdb8490ea06be2e89efd953f"
format_stamp: "Formatted at 2026-08-11 17:14:09 on dist-test-slave-fd7l"
server_key: "6956fadbe57c0375896d1bc74bac003a"
server_key_iv: "08a0bcb3a467873d2790672b09281364"
server_key_version: "encryptionkey@0"
I20260811 17:14:09.614521 25364 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:14:09.617604 25382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:09.618991 25364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:09.619158 25364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "09d7564abdb8490ea06be2e89efd953f"
format_stamp: "Formatted at 2026-08-11 17:14:09 on dist-test-slave-fd7l"
server_key: "6956fadbe57c0375896d1bc74bac003a"
server_key_iv: "08a0bcb3a467873d2790672b09281364"
server_key_version: "encryptionkey@0"
I20260811 17:14:09.619295 25364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:09.631891 25364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:09.635346 25364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:09.635583 25364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:09.636242 25364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:09.637184 25364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:09.637271 25364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:09.637368 25364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:09.637418 25364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:09.647192 25364 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:39847
I20260811 17:14:09.647212 25519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:39847 every 8 connection(s)
I20260811 17:14:09.648176 25364 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:14:09.650329 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 25364
I20260811 17:14:09.650463 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:14:09.650761 21722 external_mini_cluster.cc:1428] Setting key 437cd0f1cf56295fa34731ed61862a10
Aug 11 17:14:09 dist-test-slave-fd7l krb5kdc[25241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468449, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.54.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.130@KRBTEST.COM" created.
I20260811 17:14:09.665462 25338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:09.650178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:33599 (local address 127.21.54.190:42029)
0811 17:14:09.650360 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:09.650365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:09.651363 (+ 998us) server_negotiation.cc:407] Connection header received
0811 17:14:09.652984 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:09.652990 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:09.653056 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:09.653157 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:09.654528 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:09.655428 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:09.656749 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:09.657199 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:09.657338 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:09.660707 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:09.660712 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:09.660716 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:09.660757 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:09.662882 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:09.663506 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:09.663510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:09.663511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:09.663636 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:09.664025 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:09.664028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:09.664030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:09.664256 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:09.664373 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:09.665065 (+ 692us) server_negotiation.cc:299] Negotiation successful
0811 17:14:09.665250 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 17:14:09.666422 25523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:09.650635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:42029 (local address 127.21.54.129:33599)
0811 17:14:09.651137 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:09.651197 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:09.652750 (+ 1553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:09.653368 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:09.653376 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:09.653819 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:09.654368 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:09.654383 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:09.655587 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:09.655589 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:09.656622 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:09.656633 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:09.657392 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:09.657395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:09.657702 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:09.658464 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:09.658531 (+ 67us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:09.660425 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:09.663044 (+ 2619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:09.663052 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:09.663073 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:09.663376 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:09.663737 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:09.663741 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:09.663743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:09.663914 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:09.664373 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:09.664383 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:09.664771 (+ 388us) client_negotiation.cc:769] Sending connection context
0811 17:14:09.664979 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 17:14:09.665432 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":147,"threads_started":1}
I20260811 17:14:09.669497 25520 heartbeater.cc:347] Connected to a master server at 127.21.54.190:42029
I20260811 17:14:09.669797 25520 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:09.670383 25520 heartbeater.cc:511] Master 127.21.54.190:42029 requested a full tablet report, sending...
I20260811 17:14:09.672313 25287 ts_manager.cc:194] Registered new tserver with Master: 09d7564abdb8490ea06be2e89efd953f (127.21.54.129:39847)
I20260811 17:14:09.673542 25287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:33599
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
I20260811 17:14:09.686194 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:42029
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:33439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:09.810024 25526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:09.810328 25526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:09.810374 25526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:09.814504 25526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:09.814584 25526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:09.814646 25526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:14:09.821094 25526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:33439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:42029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:09.822275 25526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:09.823427 25526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:09.834209 25540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:09.834192 25536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:09.834192 25537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:09.834759 25526 server_base.cc:1034] running on GCE node
I20260811 17:14:09.835228 25526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:09.835927 25526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:09.837160 25526 hybrid_clock.cc:648] HybridClock initialized: now 1786468449837115 us; error 50 us; skew 500 ppm
Aug 11 17:14:09 dist-test-slave-fd7l krb5kdc[25241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468449, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:09.840348 25526 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:14:09.841418 25526 webserver.cc:460] Webserver started at http://127.21.54.130:46403/ using document root <none> and password file <none>
I20260811 17:14:09.842051 25526 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:09.842168 25526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:09.842420 25526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:09.844848 25526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "867faaa6719f43b4b2a6a4b454a51716"
format_stamp: "Formatted at 2026-08-11 17:14:09 on dist-test-slave-fd7l"
server_key: "f93aaa64b0e45999f66833195d8d667d"
server_key_iv: "bb06171992cac937022a075081a007e5"
server_key_version: "encryptionkey@0"
I20260811 17:14:09.845463 25526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "867faaa6719f43b4b2a6a4b454a51716"
format_stamp: "Formatted at 2026-08-11 17:14:09 on dist-test-slave-fd7l"
server_key: "f93aaa64b0e45999f66833195d8d667d"
server_key_iv: "bb06171992cac937022a075081a007e5"
server_key_version: "encryptionkey@0"
I20260811 17:14:09.849818 25526 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:14:09.852675 25551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:09.854076 25526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 17:14:09.854249 25526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "867faaa6719f43b4b2a6a4b454a51716"
format_stamp: "Formatted at 2026-08-11 17:14:09 on dist-test-slave-fd7l"
server_key: "f93aaa64b0e45999f66833195d8d667d"
server_key_iv: "bb06171992cac937022a075081a007e5"
server_key_version: "encryptionkey@0"
I20260811 17:14:09.854387 25526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:09.890913 25526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:09.895308 25526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:09.895598 25526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:09.896384 25526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:09.898011 25526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:09.898106 25526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:09.898195 25526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:09.898248 25526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:09.909307 25526 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:44363
I20260811 17:14:09.909327 25685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:44363 every 8 connection(s)
I20260811 17:14:09.910424 25526 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:14:09.915584 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 25526
I20260811 17:14:09.915752 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:14:09.916118 21722 external_mini_cluster.cc:1428] Setting key d310804e9ace73b3dc42193377a74c57
Aug 11 17:14:09 dist-test-slave-fd7l krb5kdc[25241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468449, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.54.131@KRBTEST.COM; defaulting to no policy
I20260811 17:14:09.928200 25338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:09.912737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:33211 (local address 127.21.54.190:42029)
0811 17:14:09.912953 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:09.912957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:09.914008 (+ 1051us) server_negotiation.cc:407] Connection header received
0811 17:14:09.915774 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:09.915778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:09.915849 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:09.915979 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:09.917678 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:09.918452 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:09.919954 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:09.920349 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:09.920461 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:09.924168 (+ 3707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:09.924171 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:09.924173 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:09.924201 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:09.925846 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:09.926477 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:09.926481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:09.926482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:09.926574 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:09.926962 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:09.926966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:09.926968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:09.927183 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:09.927298 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:09.927875 (+ 577us) server_negotiation.cc:299] Negotiation successful
0811 17:14:09.928028 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 17:14:09.929185 25690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:09.913212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:42029 (local address 127.21.54.130:33211)
0811 17:14:09.913813 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:09.913863 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:09.915391 (+ 1528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:09.916229 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:09.916240 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:09.916783 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:09.917494 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:09.917529 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:09.918637 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:09.918640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:09.919780 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:09.919791 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:09.920687 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:09.920694 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:09.920965 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:09.921786 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:09.921814 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:09.923841 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:09.926014 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:09.926021 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:09.926045 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:09.926345 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:09.926676 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:09.926679 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:09.926681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:09.926844 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:09.927298 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:09.927306 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:09.927604 (+ 298us) client_negotiation.cc:769] Sending connection context
0811 17:14:09.927816 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 17:14:09.928202 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":133,"threads_started":1}
I20260811 17:14:09.930869 25688 heartbeater.cc:347] Connected to a master server at 127.21.54.190:42029
I20260811 17:14:09.931178 25688 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.131@KRBTEST.COM" created.
I20260811 17:14:09.931836 25688 heartbeater.cc:511] Master 127.21.54.190:42029 requested a full tablet report, sending...
I20260811 17:14:09.933286 25287 ts_manager.cc:194] Registered new tserver with Master: 867faaa6719f43b4b2a6a4b454a51716 (127.21.54.130:44363)
I20260811 17:14:09.934026 25287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:33211
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
I20260811 17:14:09.953007 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:42029
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:33439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:10.076161 25693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:10.076545 25693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:10.076670 25693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:10.080880 25693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:10.080997 25693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:10.081105 25693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:14:10.086922 25693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:33439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:42029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:10.088181 25693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:10.089346 25693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:10.100076 25698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:10.100096 25699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:10.100080 25703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:10.100363 25693 server_base.cc:1034] running on GCE node
I20260811 17:14:10.100984 25693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:10.101778 25693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:10.102969 25693 hybrid_clock.cc:648] HybridClock initialized: now 1786468450102904 us; error 77 us; skew 500 ppm
Aug 11 17:14:10 dist-test-slave-fd7l krb5kdc[25241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468450, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:10.106297 25693 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:14:10.107308 25693 webserver.cc:460] Webserver started at http://127.21.54.131:41161/ using document root <none> and password file <none>
I20260811 17:14:10.107964 25693 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:10.108031 25693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:10.108269 25693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:10.110594 25693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "5953e57989634fe8826771743f79f4dd"
format_stamp: "Formatted at 2026-08-11 17:14:10 on dist-test-slave-fd7l"
server_key: "6efc45ae0cac572db2ed621eea53e4ed"
server_key_iv: "86b9bd3e101698350dda54b946a8295a"
server_key_version: "encryptionkey@0"
I20260811 17:14:10.111207 25693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "5953e57989634fe8826771743f79f4dd"
format_stamp: "Formatted at 2026-08-11 17:14:10 on dist-test-slave-fd7l"
server_key: "6efc45ae0cac572db2ed621eea53e4ed"
server_key_iv: "86b9bd3e101698350dda54b946a8295a"
server_key_version: "encryptionkey@0"
I20260811 17:14:10.115502 25693 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 17:14:10.118657 25712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:10.120144 25693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:14:10.120302 25693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "5953e57989634fe8826771743f79f4dd"
format_stamp: "Formatted at 2026-08-11 17:14:10 on dist-test-slave-fd7l"
server_key: "6efc45ae0cac572db2ed621eea53e4ed"
server_key_iv: "86b9bd3e101698350dda54b946a8295a"
server_key_version: "encryptionkey@0"
I20260811 17:14:10.120432 25693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:10.134745 25693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:10.138607 25693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:10.138854 25693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:10.139546 25693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:10.140496 25693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:10.140681 25693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:10.140758 25693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:10.140811 25693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:10.151281 25693 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:42639
I20260811 17:14:10.151309 25844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:42639 every 8 connection(s)
I20260811 17:14:10.152380 25693 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:14:10.161926 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 25693
I20260811 17:14:10.162113 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:14:10.162544 21722 external_mini_cluster.cc:1428] Setting key 44d66f8426867d0798c74834c079cec7
Aug 11 17:14:10 dist-test-slave-fd7l krb5kdc[25241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468450, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:10.169006 25338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:10.154778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:33445 (local address 127.21.54.190:42029)
0811 17:14:10.154966 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:10.154970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:10.156114 (+ 1144us) server_negotiation.cc:407] Connection header received
0811 17:14:10.157683 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:10.157686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:10.157756 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:10.157904 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:10.159433 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:10.160038 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:10.161375 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:10.161611 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:10.161739 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:10.164974 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:10.164979 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:10.164982 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:10.165016 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:10.166777 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:10.167416 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:10.167419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:10.167420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:10.167504 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:10.167839 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:10.167841 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:10.167843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:10.168040 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:10.168140 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:10.168757 (+ 617us) server_negotiation.cc:299] Negotiation successful
0811 17:14:10.168880 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 17:14:10.169991 25847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:10.155245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:42029 (local address 127.21.54.131:33445)
0811 17:14:10.155893 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:10.155942 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:10.157407 (+ 1465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:10.158161 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:10.158170 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:10.158692 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:10.159234 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:10.159249 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:10.160202 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:10.160207 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:10.161219 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:10.161231 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:10.161790 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:10.161794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:10.161987 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:10.162783 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:10.162802 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:10.164789 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:10.166921 (+ 2132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:10.166927 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:10.166949 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:10.167285 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:10.167592 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:10.167594 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:10.167595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:10.167743 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:10.168139 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:10.168144 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:10.168425 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 17:14:10.168679 (+ 254us) client_negotiation.cc:240] Negotiation successful
0811 17:14:10.168982 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":147,"threads_started":1}
I20260811 17:14:10.171379 25845 heartbeater.cc:347] Connected to a master server at 127.21.54.190:42029
I20260811 17:14:10.171761 25845 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:10.172359 25845 heartbeater.cc:511] Master 127.21.54.190:42029 requested a full tablet report, sending...
I20260811 17:14:10.173821 25287 ts_manager.cc:194] Registered new tserver with Master: 5953e57989634fe8826771743f79f4dd (127.21.54.131:42639)
I20260811 17:14:10.174561 25287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:33445
I20260811 17:14:10.177692 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 17:14:10 dist-test-slave-fd7l krb5kdc[25241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468450, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 17:14:10 dist-test-slave-fd7l krb5kdc[25241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468450, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:10.209529 25338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:10.198845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34978 (local address 127.21.54.190:42029)
0811 17:14:10.199117 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:10.199123 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:10.199296 (+ 173us) server_negotiation.cc:407] Connection header received
0811 17:14:10.199498 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:10.199507 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:10.199558 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:10.199665 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:10.200532 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:10.201179 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:10.202191 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:10.202451 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:10.202597 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:10.205611 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:10.205615 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:10.205618 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:10.205662 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:10.207412 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:10.207936 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:10.207941 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:10.207943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:10.208049 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:10.208379 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:10.208383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:10.208385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:10.208729 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:10.208892 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:10.209197 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 17:14:10.209344 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
W20260811 17:14:10.210000 25287 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34978
I20260811 17:14:10.211161 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 25364
I20260811 17:14:10.221815 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 25526
I20260811 17:14:10.231148 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 25693
I20260811 17:14:10.241840 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 25256
2026-08-11T17:14:10Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2424 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:14:10 dist-test-slave-fd7l krb5kdc[25860](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:14:10 dist-test-slave-fd7l krb5kdc[25860](info): set up 2 sockets
Aug 11 17:14:10 dist-test-slave-fd7l krb5kdc[25860](info): commencing operation
krb5kdc: starting...
W20260811 17:14:11.433719 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:14:11 dist-test-slave-fd7l krb5kdc[25860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:14:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:14:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:14:11.593142 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:33257
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:42747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:33257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:11.715672 25875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:11.716073 25875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:11.716145 25875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:11.720732 25875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:11.720889 25875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:11.720933 25875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:11.720969 25875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:11.721000 25875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:14:11.726979 25875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:42747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:33257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:33257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:11.728324 25875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:11.729501 25875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:11.739220 25881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:11.739243 25884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:11.739243 25882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:11.739480 25875 server_base.cc:1034] running on GCE node
I20260811 17:14:11.740330 25875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:11.741662 25875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:11.742938 25875 hybrid_clock.cc:648] HybridClock initialized: now 1786468451742914 us; error 38 us; skew 500 ppm
Aug 11 17:14:11 dist-test-slave-fd7l krb5kdc[25860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468451, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:11.746246 25875 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:14:11.747340 25875 webserver.cc:460] Webserver started at http://127.21.54.190:39389/ using document root <none> and password file <none>
I20260811 17:14:11.747943 25875 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:11.748009 25875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:11.748203 25875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:11.750458 25875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "1f91e93b718741a3bdf973d2181fd236"
format_stamp: "Formatted at 2026-08-11 17:14:11 on dist-test-slave-fd7l"
server_key: "9ecce3a971fd0e777cda12ea5b9800e5"
server_key_iv: "e138da0c850ee4af74bc816544d9a1b2"
server_key_version: "encryptionkey@0"
I20260811 17:14:11.751009 25875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "1f91e93b718741a3bdf973d2181fd236"
format_stamp: "Formatted at 2026-08-11 17:14:11 on dist-test-slave-fd7l"
server_key: "9ecce3a971fd0e777cda12ea5b9800e5"
server_key_iv: "e138da0c850ee4af74bc816544d9a1b2"
server_key_version: "encryptionkey@0"
I20260811 17:14:11.755165 25875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:14:11.757952 25894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:11.759488 25875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:14:11.759737 25875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "1f91e93b718741a3bdf973d2181fd236"
format_stamp: "Formatted at 2026-08-11 17:14:11 on dist-test-slave-fd7l"
server_key: "9ecce3a971fd0e777cda12ea5b9800e5"
server_key_iv: "e138da0c850ee4af74bc816544d9a1b2"
server_key_version: "encryptionkey@0"
I20260811 17:14:11.759868 25875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:11.786427 25875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:11.790385 25875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:11.790630 25875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:11.799387 25875 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:33257
I20260811 17:14:11.799453 25952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:33257 every 8 connection(s)
I20260811 17:14:11.800464 25875 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:14:11.802341 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 25875
I20260811 17:14:11.802455 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:14:11.802827 21722 external_mini_cluster.cc:1428] Setting key b4e6c9835bd7245d56f038c071b22acf
I20260811 17:14:11.804243 25954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:11.813058 25954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236: Bootstrap starting.
Aug 11 17:14:11 dist-test-slave-fd7l krb5kdc[25860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:11.816145 25954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:11.817199 25954 log.cc:824] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:11.819942 25954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236: No bootstrap required, opened a new log
I20260811 17:14:11.821296 25957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:11.804578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58706 (local address 127.21.54.190:33257)
0811 17:14:11.805092 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:11.805125 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:11.805178 (+ 53us) server_negotiation.cc:407] Connection header received
0811 17:14:11.806248 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:11.806277 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:11.806754 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:11.807218 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:11.808084 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:11.809912 (+ 1828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:11.811113 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:11.812014 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:11.812240 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:11.815108 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:11.815116 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:11.815135 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:11.815183 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:11.817513 (+ 2330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:11.818156 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:11.818203 (+ 47us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:11.818211 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:11.818391 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:11.818799 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:11.818802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:11.818803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:11.819392 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:11.819579 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:11.819922 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 17:14:11.820266 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":135,"threads_started":1}
I20260811 17:14:11.823733 25954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f91e93b718741a3bdf973d2181fd236" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 33257 } }
I20260811 17:14:11.824046 25954 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:11.824107 25954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f91e93b718741a3bdf973d2181fd236, State: Initialized, Role: FOLLOWER
I20260811 17:14:11.824609 25954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f91e93b718741a3bdf973d2181fd236" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 33257 } }
I20260811 17:14:11.824748 25954 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:14:11.824792 25954 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:14:11.824867 25954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:11.825863 25954 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f91e93b718741a3bdf973d2181fd236" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 33257 } }
I20260811 17:14:11.826186 25954 leader_election.cc:302] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f91e93b718741a3bdf973d2181fd236; no voters:
I20260811 17:14:11.826501 25954 leader_election.cc:288] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:14:11.826754 25959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:11.827204 25959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236 [term 1 LEADER]: Becoming Leader. State: Replica: 1f91e93b718741a3bdf973d2181fd236, State: Running, Role: LEADER
I20260811 17:14:11.827555 25959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f91e93b718741a3bdf973d2181fd236" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 33257 } }
I20260811 17:14:11.828014 25954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:11.829404 25963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f91e93b718741a3bdf973d2181fd236. Latest consensus state: current_term: 1 leader_uuid: "1f91e93b718741a3bdf973d2181fd236" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f91e93b718741a3bdf973d2181fd236" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 33257 } } }
I20260811 17:14:11.829556 25963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:11.829942 25960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f91e93b718741a3bdf973d2181fd236" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f91e93b718741a3bdf973d2181fd236" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 33257 } } }
I20260811 17:14:11.830065 25960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236 [sys.catalog]: This master's current role is: LEADER
W20260811 17:14:11.832904 25978 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:14:11.832978 25978 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:14:11.833065 25973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:14:11.833784 25973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:14:11.839504 25973 catalog_manager.cc:1347] Generated new cluster ID: f15c3abb0935499894765f54c1d2c5c4
I20260811 17:14:11.839586 25973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:14:11.856559 25973 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:14:11.857695 25973 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:14:11.868264 25973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f91e93b718741a3bdf973d2181fd236: Generated new TSK 0
I20260811 17:14:11.869393 25973 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
I20260811 17:14:11.913443 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:33257
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:42747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:12.037758 25986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:12.038074 25986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:12.038157 25986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:12.042695 25986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:12.042826 25986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:12.042925 25986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:14:12.048664 25986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:42747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:33257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:12.049959 25986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:12.051187 25986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:12.062101 25991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:12.062115 25996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:12.062095 25993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:12.062484 25986 server_base.cc:1034] running on GCE node
I20260811 17:14:12.063167 25986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:12.064014 25986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:12.065229 25986 hybrid_clock.cc:648] HybridClock initialized: now 1786468452065182 us; error 64 us; skew 500 ppm
Aug 11 17:14:12 dist-test-slave-fd7l krb5kdc[25860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468452, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:12.068457 25986 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:14:12.069644 25986 webserver.cc:460] Webserver started at http://127.21.54.129:38293/ using document root <none> and password file <none>
I20260811 17:14:12.070327 25986 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:12.070396 25986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:12.070657 25986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:12.073056 25986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "740ef25baecb45a0a695cc949eb7d051"
format_stamp: "Formatted at 2026-08-11 17:14:12 on dist-test-slave-fd7l"
server_key: "59a680a6d3e4ed042c74047449c4b2ab"
server_key_iv: "1b79d81f70ce3719b0b5b7e870674e94"
server_key_version: "encryptionkey@0"
I20260811 17:14:12.073698 25986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "740ef25baecb45a0a695cc949eb7d051"
format_stamp: "Formatted at 2026-08-11 17:14:12 on dist-test-slave-fd7l"
server_key: "59a680a6d3e4ed042c74047449c4b2ab"
server_key_iv: "1b79d81f70ce3719b0b5b7e870674e94"
server_key_version: "encryptionkey@0"
I20260811 17:14:12.077840 25986 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 17:14:12.080823 26003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:12.082481 25986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:12.082603 25986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "740ef25baecb45a0a695cc949eb7d051"
format_stamp: "Formatted at 2026-08-11 17:14:12 on dist-test-slave-fd7l"
server_key: "59a680a6d3e4ed042c74047449c4b2ab"
server_key_iv: "1b79d81f70ce3719b0b5b7e870674e94"
server_key_version: "encryptionkey@0"
I20260811 17:14:12.082775 25986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:12.101467 25986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:12.105098 25986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:12.105329 25986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:12.106113 25986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:12.107035 25986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:12.107092 25986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:12.107170 25986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:12.107203 25986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:12.117475 25986 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:44827
I20260811 17:14:12.117511 26136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:44827 every 8 connection(s)
I20260811 17:14:12.118471 25986 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:14:12.122624 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 25986
I20260811 17:14:12.122781 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:14:12.123149 21722 external_mini_cluster.cc:1428] Setting key 738caa8cf9cec72e065e2e5e63ee9881
Aug 11 17:14:12 dist-test-slave-fd7l krb5kdc[25860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468452, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.54.130@KRBTEST.COM; defaulting to no policy
I20260811 17:14:12.136627 25957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:12.120695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:60039 (local address 127.21.54.190:33257)
0811 17:14:12.120853 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:12.120857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:12.121896 (+ 1039us) server_negotiation.cc:407] Connection header received
0811 17:14:12.123857 (+ 1961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:12.123861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:12.123917 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:12.123994 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:12.125674 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:12.126292 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:12.127900 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:12.128227 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:12.128353 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:12.132003 (+ 3650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:12.132008 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:12.132011 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:12.132063 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:12.133923 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:12.134582 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:12.134594 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:12.134596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:12.134698 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:12.135119 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:12.135122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:12.135124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:12.135418 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:12.135526 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:12.136249 (+ 723us) server_negotiation.cc:299] Negotiation successful
0811 17:14:12.136409 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 17:14:12.137562 26139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:12.121150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:33257 (local address 127.21.54.129:60039)
0811 17:14:12.121680 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:12.121730 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:12.123567 (+ 1837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:12.124221 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:12.124228 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:12.124728 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:12.125443 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:12.125469 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:12.126449 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:12.126455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:12.127746 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:12.127757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:12.128406 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:12.128411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:12.128675 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:12.129511 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:12.129536 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:12.131744 (+ 2208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:12.134109 (+ 2365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:12.134117 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:12.134140 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:12.134445 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:12.134809 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:12.134812 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:12.134815 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:12.134983 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:12.135542 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:12.135551 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:12.135929 (+ 378us) client_negotiation.cc:769] Sending connection context
0811 17:14:12.136183 (+ 254us) client_negotiation.cc:240] Negotiation successful
0811 17:14:12.136608 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":129,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.130@KRBTEST.COM" created.
I20260811 17:14:12.139065 26137 heartbeater.cc:347] Connected to a master server at 127.21.54.190:33257
I20260811 17:14:12.139387 26137 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:12.140084 26137 heartbeater.cc:511] Master 127.21.54.190:33257 requested a full tablet report, sending...
I20260811 17:14:12.142352 25913 ts_manager.cc:194] Registered new tserver with Master: 740ef25baecb45a0a695cc949eb7d051 (127.21.54.129:44827)
I20260811 17:14:12.143826 25913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:60039
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
I20260811 17:14:12.161029 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:33257
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:42747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:12.281773 26143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:12.282095 26143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:12.282168 26143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:12.286571 26143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:12.286654 26143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:12.286747 26143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:14:12.292543 26143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:42747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:33257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:12.293838 26143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:12.294986 26143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:12.305896 26150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:12.305855 26149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:12.305874 26152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:12.306198 26143 server_base.cc:1034] running on GCE node
I20260811 17:14:12.306962 26143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:12.307828 26143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:12.309047 26143 hybrid_clock.cc:648] HybridClock initialized: now 1786468452309025 us; error 34 us; skew 500 ppm
Aug 11 17:14:12 dist-test-slave-fd7l krb5kdc[25860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468452, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:12.312670 26143 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:14:12.313839 26143 webserver.cc:460] Webserver started at http://127.21.54.130:41315/ using document root <none> and password file <none>
I20260811 17:14:12.314443 26143 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:12.314508 26143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:12.314702 26143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:12.317149 26143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "e7e54bc92f09486a8211de628623e42d"
format_stamp: "Formatted at 2026-08-11 17:14:12 on dist-test-slave-fd7l"
server_key: "a09018ba7b9c793319fabecd1dfe4975"
server_key_iv: "8ad66a399fb9ac13220cd7e4aa3da499"
server_key_version: "encryptionkey@0"
I20260811 17:14:12.317731 26143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "e7e54bc92f09486a8211de628623e42d"
format_stamp: "Formatted at 2026-08-11 17:14:12 on dist-test-slave-fd7l"
server_key: "a09018ba7b9c793319fabecd1dfe4975"
server_key_iv: "8ad66a399fb9ac13220cd7e4aa3da499"
server_key_version: "encryptionkey@0"
I20260811 17:14:12.322247 26143 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:14:12.325218 26161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:12.326664 26143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:12.326784 26143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "e7e54bc92f09486a8211de628623e42d"
format_stamp: "Formatted at 2026-08-11 17:14:12 on dist-test-slave-fd7l"
server_key: "a09018ba7b9c793319fabecd1dfe4975"
server_key_iv: "8ad66a399fb9ac13220cd7e4aa3da499"
server_key_version: "encryptionkey@0"
I20260811 17:14:12.326887 26143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:12.339274 26143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:12.343007 26143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:12.343187 26143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:12.343783 26143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:12.344782 26143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:12.344836 26143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:12.344879 26143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:12.344895 26143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:12.354918 26143 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:39431
I20260811 17:14:12.354938 26291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:39431 every 8 connection(s)
I20260811 17:14:12.355933 26143 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:14:12.359866 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 26143
I20260811 17:14:12.360004 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:14:12.360299 21722 external_mini_cluster.cc:1428] Setting key 8aba329051b6531933d094e737d4635f
Aug 11 17:14:12 dist-test-slave-fd7l krb5kdc[25860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468452, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.54.131@KRBTEST.COM; defaulting to no policy
I20260811 17:14:12.373446 25957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:12.358042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:52797 (local address 127.21.54.190:33257)
0811 17:14:12.358207 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:12.358212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:12.359161 (+ 949us) server_negotiation.cc:407] Connection header received
0811 17:14:12.360990 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:12.360994 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:12.361066 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:12.361175 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:12.363042 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:12.363873 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:12.365366 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:12.365752 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:12.365886 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:12.369383 (+ 3497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:12.369387 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:12.369390 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:12.369422 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:12.370984 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:12.371635 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:12.371639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:12.371641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:12.371735 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:12.372125 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:12.372128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:12.372130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:12.372335 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:12.372445 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:12.373119 (+ 674us) server_negotiation.cc:299] Negotiation successful
0811 17:14:12.373269 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 17:14:12.374456 26297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:12.358404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:33257 (local address 127.21.54.130:52797)
0811 17:14:12.358906 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:12.358953 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:12.360631 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:12.361501 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:12.361511 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:12.362270 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:12.362859 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:12.362874 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:12.364036 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:12.364039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:12.365231 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:12.365243 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:12.365960 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:12.365969 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:12.366282 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:12.367069 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:12.367100 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:12.369142 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:12.371170 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:12.371177 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:12.371197 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:12.371501 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:12.371837 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:12.371840 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:12.371842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:12.372005 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:12.372458 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:12.372467 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:12.372835 (+ 368us) client_negotiation.cc:769] Sending connection context
0811 17:14:12.373052 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 17:14:12.373430 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":115,"threads_started":1}
I20260811 17:14:12.375958 26294 heartbeater.cc:347] Connected to a master server at 127.21.54.190:33257
I20260811 17:14:12.376325 26294 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.131@KRBTEST.COM" created.
I20260811 17:14:12.377086 26294 heartbeater.cc:511] Master 127.21.54.190:33257 requested a full tablet report, sending...
I20260811 17:14:12.378456 25913 ts_manager.cc:194] Registered new tserver with Master: e7e54bc92f09486a8211de628623e42d (127.21.54.130:39431)
I20260811 17:14:12.379184 25913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:52797
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
I20260811 17:14:12.400741 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:33257
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:42747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:12.552418 26301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:12.552856 26301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:12.552947 26301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:12.557360 26301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:12.557435 26301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:12.557500 26301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:14:12.563607 26301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:42747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:33257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:12.564996 26301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:12.566146 26301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:12.577500 26306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:12.577500 26307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:12.577806 26301 server_base.cc:1034] running on GCE node
W20260811 17:14:12.578047 26310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:12.578557 26301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:12.579389 26301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:12.580587 26301 hybrid_clock.cc:648] HybridClock initialized: now 1786468452580557 us; error 49 us; skew 500 ppm
Aug 11 17:14:12 dist-test-slave-fd7l krb5kdc[25860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468452, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:12.583583 26301 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:14:12.584743 26301 webserver.cc:460] Webserver started at http://127.21.54.131:43215/ using document root <none> and password file <none>
I20260811 17:14:12.585351 26301 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:12.585417 26301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:12.585652 26301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:12.587962 26301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "56b3e1588b074f508c961486e47a5364"
format_stamp: "Formatted at 2026-08-11 17:14:12 on dist-test-slave-fd7l"
server_key: "524b0f9cb7e3e16b5443c6abc4759670"
server_key_iv: "564052656d7fb0a35addc04e97a307a7"
server_key_version: "encryptionkey@0"
I20260811 17:14:12.588651 26301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "56b3e1588b074f508c961486e47a5364"
format_stamp: "Formatted at 2026-08-11 17:14:12 on dist-test-slave-fd7l"
server_key: "524b0f9cb7e3e16b5443c6abc4759670"
server_key_iv: "564052656d7fb0a35addc04e97a307a7"
server_key_version: "encryptionkey@0"
I20260811 17:14:12.592880 26301 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:14:12.595719 26318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:12.597227 26301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:12.597380 26301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "56b3e1588b074f508c961486e47a5364"
format_stamp: "Formatted at 2026-08-11 17:14:12 on dist-test-slave-fd7l"
server_key: "524b0f9cb7e3e16b5443c6abc4759670"
server_key_iv: "564052656d7fb0a35addc04e97a307a7"
server_key_version: "encryptionkey@0"
I20260811 17:14:12.597527 26301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:12.611033 26301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:12.614715 26301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:12.614946 26301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:12.615617 26301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:12.616726 26301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:12.616778 26301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:12.616845 26301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:12.616878 26301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:12.627065 26301 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:37551
I20260811 17:14:12.627084 26460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:37551 every 8 connection(s)
I20260811 17:14:12.628135 26301 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:14:12.631170 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 26301
I20260811 17:14:12.631330 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:14:12.631675 21722 external_mini_cluster.cc:1428] Setting key 786125b69dc9cb417e69ec81ee5fbc5a
Aug 11 17:14:12 dist-test-slave-fd7l krb5kdc[25860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468452, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:12.644284 25957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:12.630248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:60305 (local address 127.21.54.190:33257)
0811 17:14:12.630395 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:12.630399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:12.631537 (+ 1138us) server_negotiation.cc:407] Connection header received
0811 17:14:12.633377 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:12.633381 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:12.633447 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:12.633543 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:12.635061 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:12.635674 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:12.637217 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:12.637460 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:12.637624 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:12.640389 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:12.640398 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:12.640400 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:12.640431 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:12.641953 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:12.642556 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:12.642560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:12.642562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:12.642651 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:12.643040 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:12.643044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:12.643046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:12.643258 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:12.643377 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:12.643948 (+ 571us) server_negotiation.cc:299] Negotiation successful
0811 17:14:12.644103 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:14:12.645421 26464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:12.630674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:33257 (local address 127.21.54.131:60305)
0811 17:14:12.631291 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:12.631353 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:12.632982 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:12.633836 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:12.633845 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:12.634296 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:12.634872 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:12.634888 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:12.635829 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:12.635833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:12.637022 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:12.637033 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:12.637690 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:12.637694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:12.637901 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:12.638594 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:12.638611 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:12.640176 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:12.642096 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:12.642102 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:12.642122 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:12.642427 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:12.642764 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:12.642767 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:12.642769 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:12.642931 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:12.643375 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:12.643382 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:12.643673 (+ 291us) client_negotiation.cc:769] Sending connection context
0811 17:14:12.643893 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 17:14:12.644261 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":180,"threads_started":1}
I20260811 17:14:12.646795 26461 heartbeater.cc:347] Connected to a master server at 127.21.54.190:33257
I20260811 17:14:12.647153 26461 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:12.647863 26461 heartbeater.cc:511] Master 127.21.54.190:33257 requested a full tablet report, sending...
I20260811 17:14:12.649233 25911 ts_manager.cc:194] Registered new tserver with Master: 56b3e1588b074f508c961486e47a5364 (127.21.54.131:37551)
I20260811 17:14:12.650077 25911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:60305
I20260811 17:14:12.658578 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 17:14:12 dist-test-slave-fd7l krb5kdc[25860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 17:14:12 dist-test-slave-fd7l krb5kdc[25860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.129@KRBTEST.COM
I20260811 17:14:12.690121 26470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:12.679100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53850 (local address 127.21.54.129:44827)
0811 17:14:12.679416 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:12.679423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:12.679456 (+ 33us) server_negotiation.cc:407] Connection header received
0811 17:14:12.679520 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:12.679525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:12.679729 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:12.679860 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:12.680759 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:12.681621 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:12.682558 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:12.682992 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:12.683091 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:12.686030 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:12.686043 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:12.686059 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:12.686110 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:12.688064 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:12.688692 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:12.688698 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:12.688699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:12.688795 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:12.689133 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:12.689135 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:12.689137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:12.689402 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:12.689566 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:12.689811 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 17:14:12.689953 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":94,"threads_started":1}
Aug 11 17:14:12 dist-test-slave-fd7l krb5kdc[25860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:12.703629 25957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:12.693628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58708 (local address 127.21.54.190:33257)
0811 17:14:12.693796 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:12.693801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:12.693920 (+ 119us) server_negotiation.cc:407] Connection header received
0811 17:14:12.694098 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:12.694101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:12.694159 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:12.694247 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:12.694931 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:12.695475 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:12.696334 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:12.696668 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:12.696784 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:12.699432 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:12.699436 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:12.699439 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:12.699477 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:12.701225 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:12.701864 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:12.701871 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:12.701876 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:12.702010 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:12.702392 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:12.702395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:12.702396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:12.702767 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:12.702892 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:12.703232 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 17:14:12.703352 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 17:14:12.704084 25911 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58708
I20260811 17:14:12.704910 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 25986
I20260811 17:14:12.714740 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 26143
I20260811 17:14:12.723706 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 26301
I20260811 17:14:12.732210 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 25875
2026-08-11T17:14:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2487 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:14:12 dist-test-slave-fd7l krb5kdc[26478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:14:12 dist-test-slave-fd7l krb5kdc[26478](info): set up 2 sockets
Aug 11 17:14:12 dist-test-slave-fd7l krb5kdc[26478](info): commencing operation
krb5kdc: starting...
W20260811 17:14:13.970328 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260811 17:14:14.194322 21722 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.224s user 0.001s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260811 17:14:14.435817 21722 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.241s user 0.001s sys 0.001s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260811 17:14:14.653169 21722 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.217s user 0.000s sys 0.003s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/test-user.keytab.
W20260811 17:14:15.301504 21722 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.648s user 0.000s sys 0.002s
Aug 11 17:14:15 dist-test-slave-fd7l krb5kdc[26478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:14:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:14:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:14:15.361681 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:43761
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:41029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:43761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:15.488787 26493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:15.489096 26493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:15.489184 26493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:15.493454 26493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:15.493535 26493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:15.493556 26493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:15.493573 26493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:15.493595 26493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:14:15.499305 26493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:41029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:43761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:43761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:15.500650 26493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:15.501796 26493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:15.511385 26499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:15.511472 26493 server_base.cc:1034] running on GCE node
W20260811 17:14:15.511385 26502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:15.511359 26500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:15.512339 26493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:15.513666 26493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:15.514899 26493 hybrid_clock.cc:648] HybridClock initialized: now 1786468455514846 us; error 69 us; skew 500 ppm
Aug 11 17:14:15 dist-test-slave-fd7l krb5kdc[26478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468455, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:15.528723 26493 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:14:15.529744 26493 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:15.529871 26493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:15.530179 26493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:15.532498 26493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "78377c4aee544fd78cb67021497c544b"
format_stamp: "Formatted at 2026-08-11 17:14:15 on dist-test-slave-fd7l"
server_key: "453490c7c2b28a928685e0883c0af942"
server_key_iv: "64ad349dd34f8d281659de3d9b78aeb3"
server_key_version: "encryptionkey@0"
I20260811 17:14:15.533247 26493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "78377c4aee544fd78cb67021497c544b"
format_stamp: "Formatted at 2026-08-11 17:14:15 on dist-test-slave-fd7l"
server_key: "453490c7c2b28a928685e0883c0af942"
server_key_iv: "64ad349dd34f8d281659de3d9b78aeb3"
server_key_version: "encryptionkey@0"
I20260811 17:14:15.537609 26493 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:14:15.540735 26508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:15.542336 26493 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 17:14:15.542524 26493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "78377c4aee544fd78cb67021497c544b"
format_stamp: "Formatted at 2026-08-11 17:14:15 on dist-test-slave-fd7l"
server_key: "453490c7c2b28a928685e0883c0af942"
server_key_iv: "64ad349dd34f8d281659de3d9b78aeb3"
server_key_version: "encryptionkey@0"
I20260811 17:14:15.542668 26493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:15.558137 26493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:15.562135 26493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:15.562398 26493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:15.570214 26493 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:43761
I20260811 17:14:15.570544 26571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:43761 every 8 connection(s)
I20260811 17:14:15.571067 26493 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:14:15.574434 26572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:15.580861 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 26493
I20260811 17:14:15.580861 26572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b: Bootstrap starting.
I20260811 17:14:15.580986 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:14:15.581286 21722 external_mini_cluster.cc:1428] Setting key 6f1ebaede898a0b8acafcaa21620d368
I20260811 17:14:15.583861 26572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:15.584741 26572 log.cc:824] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:15.586879 26572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b: No bootstrap required, opened a new log
Aug 11 17:14:15 dist-test-slave-fd7l krb5kdc[26478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:15.591506 26572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78377c4aee544fd78cb67021497c544b" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43761 } }
I20260811 17:14:15.591825 26572 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:15.591908 26572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78377c4aee544fd78cb67021497c544b, State: Initialized, Role: FOLLOWER
I20260811 17:14:15.592581 26572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78377c4aee544fd78cb67021497c544b" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43761 } }
I20260811 17:14:15.592789 26572 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:14:15.592898 26572 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:14:15.593019 26572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:15.594246 26572 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78377c4aee544fd78cb67021497c544b" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43761 } }
I20260811 17:14:15.594682 26572 leader_election.cc:302] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78377c4aee544fd78cb67021497c544b; no voters:
I20260811 17:14:15.595139 26572 leader_election.cc:288] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:14:15.595304 26578 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:15.595583 26578 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b [term 1 LEADER]: Becoming Leader. State: Replica: 78377c4aee544fd78cb67021497c544b, State: Running, Role: LEADER
I20260811 17:14:15.595983 26578 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78377c4aee544fd78cb67021497c544b" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43761 } }
I20260811 17:14:15.596781 26572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:15.597846 26580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78377c4aee544fd78cb67021497c544b. Latest consensus state: current_term: 1 leader_uuid: "78377c4aee544fd78cb67021497c544b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78377c4aee544fd78cb67021497c544b" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43761 } } }
I20260811 17:14:15.597977 26580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:15.598320 26576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:15.582822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44836 (local address 127.21.54.190:43761)
0811 17:14:15.583299 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:15.583314 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:15.583350 (+ 36us) server_negotiation.cc:407] Connection header received
0811 17:14:15.584097 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:15.584116 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:15.584450 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:15.584869 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:15.585731 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:15.586723 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:15.587661 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:15.588162 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:15.588354 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:15.591391 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:15.591397 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:15.591413 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:15.591450 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:15.593676 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:15.594305 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:15.594310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:15.594316 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:15.594500 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:15.594944 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:15.594948 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:15.594950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:15.595606 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:15.595830 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:15.596735 (+ 905us) server_negotiation.cc:299] Negotiation successful
0811 17:14:15.597101 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":156,"threads_started":1}
I20260811 17:14:15.598335 26579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78377c4aee544fd78cb67021497c544b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78377c4aee544fd78cb67021497c544b" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43761 } } }
I20260811 17:14:15.598629 26579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:15.599697 26589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:14:15.603188 26589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:14:15.609493 26589 catalog_manager.cc:1347] Generated new cluster ID: c37f85c5f0ba415582faec5a84d7fb66
I20260811 17:14:15.609598 26589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:14:15.622570 26589 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:14:15.623651 26589 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:14:15.633284 26589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b: Generated new TSK 0
I20260811 17:14:15.634196 26589 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
I20260811 17:14:15.676743 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:43761
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:41029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:15.802296 26603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:15.802629 26603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:15.802680 26603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:15.807143 26603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:15.807230 26603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:15.807300 26603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:14:15.813024 26603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:41029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:43761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:15.814335 26603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:15.815551 26603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:15.826412 26608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:15.826437 26611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:15.826437 26609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:15.826900 26603 server_base.cc:1034] running on GCE node
I20260811 17:14:15.827385 26603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:15.828106 26603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:15.829314 26603 hybrid_clock.cc:648] HybridClock initialized: now 1786468455829266 us; error 61 us; skew 500 ppm
Aug 11 17:14:15 dist-test-slave-fd7l krb5kdc[26478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468455, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:15.832373 26603 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:14:15.833269 26603 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:15.833376 26603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:15.833606 26603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:15.835867 26603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "03e1d079e0734d7bbb8a048e25d9dca5"
format_stamp: "Formatted at 2026-08-11 17:14:15 on dist-test-slave-fd7l"
server_key: "a5ea91b17710d18c56d9b02253047299"
server_key_iv: "b38e48adf6db58c03c9ff5bd07dec5ba"
server_key_version: "encryptionkey@0"
I20260811 17:14:15.836601 26603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "03e1d079e0734d7bbb8a048e25d9dca5"
format_stamp: "Formatted at 2026-08-11 17:14:15 on dist-test-slave-fd7l"
server_key: "a5ea91b17710d18c56d9b02253047299"
server_key_iv: "b38e48adf6db58c03c9ff5bd07dec5ba"
server_key_version: "encryptionkey@0"
I20260811 17:14:15.840972 26603 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:14:15.843902 26619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:15.845718 26603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:15.845909 26603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "03e1d079e0734d7bbb8a048e25d9dca5"
format_stamp: "Formatted at 2026-08-11 17:14:15 on dist-test-slave-fd7l"
server_key: "a5ea91b17710d18c56d9b02253047299"
server_key_iv: "b38e48adf6db58c03c9ff5bd07dec5ba"
server_key_version: "encryptionkey@0"
I20260811 17:14:15.846065 26603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:15.859405 26603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:15.863001 26603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:15.863240 26603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:15.863924 26603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:15.864907 26603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:15.864984 26603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:15.865052 26603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:15.865103 26603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:15.875602 26603 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:44835
I20260811 17:14:15.875798 26755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:44835 every 8 connection(s)
I20260811 17:14:15.876340 26603 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:14:15.885927 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 26603
I20260811 17:14:15.886087 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:14:15.886446 21722 external_mini_cluster.cc:1428] Setting key 8fc0bb9b5d3afba67cf39a08792e58b3
Aug 11 17:14:15 dist-test-slave-fd7l krb5kdc[26478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468455, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:15.892685 26576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:15.878397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:37875 (local address 127.21.54.190:43761)
0811 17:14:15.878577 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:15.878582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:15.879482 (+ 900us) server_negotiation.cc:407] Connection header received
0811 17:14:15.880952 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:15.880956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:15.881022 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:15.881136 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:15.882480 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:15.883022 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:15.884228 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:15.884476 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:15.884635 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:15.888171 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:15.888176 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:15.888179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:15.888210 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:15.889916 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:15.890595 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:15.890601 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:15.890605 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:15.890733 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:15.891238 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:15.891241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:15.891242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:15.891455 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:15.891556 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:15.892279 (+ 723us) server_negotiation.cc:299] Negotiation successful
0811 17:14:15.892416 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 17:14:15.893918 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:15.878786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43761 (local address 127.21.54.129:37875)
0811 17:14:15.879301 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:15.879348 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:15.880714 (+ 1366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:15.881342 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:15.881349 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:15.881750 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:15.882303 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:15.882317 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:15.883176 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:15.883179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:15.884086 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:15.884096 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:15.884740 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:15.884744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:15.884917 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:15.885758 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:15.885787 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:15.887876 (+ 2089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:15.890076 (+ 2200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:15.890084 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:15.890106 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:15.890440 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:15.890843 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:15.890850 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:15.890852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:15.891079 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:15.891574 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:15.891584 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:15.891992 (+ 408us) client_negotiation.cc:769] Sending connection context
0811 17:14:15.892228 (+ 236us) client_negotiation.cc:240] Negotiation successful
0811 17:14:15.892681 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":133,"threads_started":1}
I20260811 17:14:15.895550 26756 heartbeater.cc:347] Connected to a master server at 127.21.54.190:43761
I20260811 17:14:15.895856 26756 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:15.896549 26756 heartbeater.cc:511] Master 127.21.54.190:43761 requested a full tablet report, sending...
I20260811 17:14:15.898777 26527 ts_manager.cc:194] Registered new tserver with Master: 03e1d079e0734d7bbb8a048e25d9dca5 (127.21.54.129:44835)
WARNING: no policy specified for kudu/127.21.54.130@KRBTEST.COM; defaulting to no policy
I20260811 17:14:15.900187 26527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:37875
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
I20260811 17:14:15.926050 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:43761
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:41029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:16.059337 26761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:16.059726 26761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:16.059824 26761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:16.064409 26761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:16.064550 26761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:16.064666 26761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:14:16.070390 26761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:41029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:43761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:16.071671 26761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:16.072805 26761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:16.083859 26767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:16.083926 26761 server_base.cc:1034] running on GCE node
W20260811 17:14:16.083859 26766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:16.083959 26769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:16.084786 26761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:16.085613 26761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:16.086863 26761 hybrid_clock.cc:648] HybridClock initialized: now 1786468456086823 us; error 50 us; skew 500 ppm
Aug 11 17:14:16 dist-test-slave-fd7l krb5kdc[26478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468456, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:16.090059 26761 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:14:16.090902 26761 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:16.090970 26761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:16.091145 26761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:16.093338 26761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "7cf9ba13b2df4d8781b21c44f4185bd6"
format_stamp: "Formatted at 2026-08-11 17:14:16 on dist-test-slave-fd7l"
server_key: "a6f9dd3c218c6892411d8cf705a8f3de"
server_key_iv: "2dd8d5354fc373f31e283c88660860ec"
server_key_version: "encryptionkey@0"
I20260811 17:14:16.093873 26761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "7cf9ba13b2df4d8781b21c44f4185bd6"
format_stamp: "Formatted at 2026-08-11 17:14:16 on dist-test-slave-fd7l"
server_key: "a6f9dd3c218c6892411d8cf705a8f3de"
server_key_iv: "2dd8d5354fc373f31e283c88660860ec"
server_key_version: "encryptionkey@0"
I20260811 17:14:16.097884 26761 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 17:14:16.100905 26777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:16.102344 26761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:14:16.102451 26761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "7cf9ba13b2df4d8781b21c44f4185bd6"
format_stamp: "Formatted at 2026-08-11 17:14:16 on dist-test-slave-fd7l"
server_key: "a6f9dd3c218c6892411d8cf705a8f3de"
server_key_iv: "2dd8d5354fc373f31e283c88660860ec"
server_key_version: "encryptionkey@0"
I20260811 17:14:16.102609 26761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:16.128943 26761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:16.132627 26761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:16.132803 26761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:16.133451 26761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:16.134378 26761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:16.134428 26761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:16.134469 26761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:16.134552 26761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:16.144630 26761 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:38671
I20260811 17:14:16.144797 26911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:38671 every 8 connection(s)
I20260811 17:14:16.145392 26761 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:14:16.155679 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 26761
I20260811 17:14:16.155800 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:14:16.156143 21722 external_mini_cluster.cc:1428] Setting key 8cd3f7160ba642b86b37a6dd2f82d9f4
Aug 11 17:14:16 dist-test-slave-fd7l krb5kdc[26478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468456, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:16.162711 26576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:16.147882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:38725 (local address 127.21.54.190:43761)
0811 17:14:16.148099 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:16.148103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:16.149137 (+ 1034us) server_negotiation.cc:407] Connection header received
0811 17:14:16.150663 (+ 1526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:16.150666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:16.150720 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:16.150809 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:16.152176 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.152862 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.154172 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.154399 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.154511 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:16.158062 (+ 3551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:16.158067 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:16.158070 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:16.158104 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:16.159754 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:16.160632 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:16.160638 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:16.160641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:16.160758 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:16.161223 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:16.161227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:16.161231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:16.161514 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:16.161627 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:16.162276 (+ 649us) server_negotiation.cc:299] Negotiation successful
0811 17:14:16.162451 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260811 17:14:16.163597 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:16.148348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43761 (local address 127.21.54.130:38725)
0811 17:14:16.148946 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:16.148994 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:16.150301 (+ 1307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:16.151024 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:16.151033 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:16.151420 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:16.152013 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:16.152028 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.153005 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.153008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:16.154019 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:16.154031 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.154657 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.154661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:16.154826 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:16.155723 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:16.155767 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:16.157727 (+ 1960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:16.159905 (+ 2178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:16.159917 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:16.159943 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:16.160444 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:16.160902 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:16.160904 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:16.160906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:16.161091 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:16.161659 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:16.161665 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:16.161957 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 17:14:16.162169 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 17:14:16.162550 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":203,"threads_started":1}
I20260811 17:14:16.165037 26912 heartbeater.cc:347] Connected to a master server at 127.21.54.190:43761
I20260811 17:14:16.165356 26912 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:16.166052 26912 heartbeater.cc:511] Master 127.21.54.190:43761 requested a full tablet report, sending...
I20260811 17:14:16.167440 26527 ts_manager.cc:194] Registered new tserver with Master: 7cf9ba13b2df4d8781b21c44f4185bd6 (127.21.54.130:38671)
I20260811 17:14:16.168180 26527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:38725
WARNING: no policy specified for kudu/127.21.54.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
I20260811 17:14:16.193114 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:43761
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:41029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:16.317179 26919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:16.317543 26919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:16.317634 26919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:16.322120 26919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:16.322265 26919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:16.322367 26919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:14:16.328358 26919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:41029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:43761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:16.329712 26919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:16.330878 26919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:16.341959 26926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:16.341946 26925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:16.341962 26929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:16.342566 26919 server_base.cc:1034] running on GCE node
I20260811 17:14:16.343150 26919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:16.343936 26919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:16.345163 26919 hybrid_clock.cc:648] HybridClock initialized: now 1786468456345123 us; error 58 us; skew 500 ppm
Aug 11 17:14:16 dist-test-slave-fd7l krb5kdc[26478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468456, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:16.348469 26919 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:14:16.349448 26919 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:16.349561 26919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:16.349802 26919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:16.352109 26919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "4a9776decd6e49a181c138d675f0b1c7"
format_stamp: "Formatted at 2026-08-11 17:14:16 on dist-test-slave-fd7l"
server_key: "64b4266e6d32681e128aa1ccb1c87ebc"
server_key_iv: "777bb47bc41de265716ccdc5f76274bc"
server_key_version: "encryptionkey@0"
I20260811 17:14:16.352859 26919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "4a9776decd6e49a181c138d675f0b1c7"
format_stamp: "Formatted at 2026-08-11 17:14:16 on dist-test-slave-fd7l"
server_key: "64b4266e6d32681e128aa1ccb1c87ebc"
server_key_iv: "777bb47bc41de265716ccdc5f76274bc"
server_key_version: "encryptionkey@0"
I20260811 17:14:16.356879 26919 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:14:16.359583 26936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:16.361052 26919 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:16.361224 26919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "4a9776decd6e49a181c138d675f0b1c7"
format_stamp: "Formatted at 2026-08-11 17:14:16 on dist-test-slave-fd7l"
server_key: "64b4266e6d32681e128aa1ccb1c87ebc"
server_key_iv: "777bb47bc41de265716ccdc5f76274bc"
server_key_version: "encryptionkey@0"
I20260811 17:14:16.361359 26919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:16.383040 26919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:16.386993 26919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:16.387226 26919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:16.387912 26919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:16.388911 26919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:16.388996 26919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:16.389068 26919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:16.389101 26919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:16.399479 26919 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:33439
I20260811 17:14:16.399557 27064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:33439 every 8 connection(s)
I20260811 17:14:16.400249 26919 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:14:16.402571 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 26919
I20260811 17:14:16.402748 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:14:16.403086 21722 external_mini_cluster.cc:1428] Setting key 4e9e0c444718423438a08be69be25496
Aug 11 17:14:16 dist-test-slave-fd7l krb5kdc[26478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468456, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:16.416579 26576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:16.402547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:42175 (local address 127.21.54.190:43761)
0811 17:14:16.402756 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:16.402762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:16.403899 (+ 1137us) server_negotiation.cc:407] Connection header received
0811 17:14:16.405573 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:16.405578 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:16.405643 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:16.405769 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:16.407341 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.407876 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.409175 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.409404 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.409494 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:16.412232 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:16.412236 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:16.412238 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:16.412265 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:16.413859 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:16.414470 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:16.414474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:16.414476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:16.414590 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:16.415115 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:16.415118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:16.415120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:16.415349 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:16.415444 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:16.416180 (+ 736us) server_negotiation.cc:299] Negotiation successful
0811 17:14:16.416350 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 17:14:16.417608 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:16.403017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43761 (local address 127.21.54.131:42175)
0811 17:14:16.403645 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:16.403694 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:16.405270 (+ 1576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:16.406088 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:16.406100 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:16.406533 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:16.407151 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:16.407188 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.407994 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.407997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:16.409033 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:16.409043 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.409553 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.409557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:16.409733 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:16.410432 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:16.410451 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:16.412012 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:16.414004 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:16.414009 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:16.414025 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:16.414313 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:16.414757 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:16.414760 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:16.414765 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:16.414965 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:16.415469 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:16.415477 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:16.415773 (+ 296us) client_negotiation.cc:769] Sending connection context
0811 17:14:16.416133 (+ 360us) client_negotiation.cc:240] Negotiation successful
0811 17:14:16.416515 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":157,"threads_started":1}
I20260811 17:14:16.418989 27065 heartbeater.cc:347] Connected to a master server at 127.21.54.190:43761
I20260811 17:14:16.419301 27065 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:16.419852 27065 heartbeater.cc:511] Master 127.21.54.190:43761 requested a full tablet report, sending...
I20260811 17:14:16.421061 26527 ts_manager.cc:194] Registered new tserver with Master: 4a9776decd6e49a181c138d675f0b1c7 (127.21.54.131:33439)
I20260811 17:14:16.421726 26527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:42175
I20260811 17:14:16.430972 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:14:16.442667 26576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:16.433623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44842 (local address 127.21.54.190:43761)
0811 17:14:16.433796 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:16.433801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:16.433994 (+ 193us) server_negotiation.cc:407] Connection header received
0811 17:14:16.434080 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:16.434083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:16.434132 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:16.434212 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:16.435026 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.435628 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.436810 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.437042 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.437138 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:16.438782 (+ 1644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:16.438785 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:16.438787 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:16.438817 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:16.440475 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:16.441043 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:16.441046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:16.441047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:16.441137 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:16.441582 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:16.441585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:16.441587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:16.441846 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:16.442007 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:16.442321 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 17:14:16.442447 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 17:14:16.447055 26527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:14:16.450700 26527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:14:16.467733 27080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:16.461886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56898 (local address 127.21.54.129:44835)
0811 17:14:16.462261 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:16.462268 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:16.462818 (+ 550us) server_negotiation.cc:407] Connection header received
0811 17:14:16.462919 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:16.462924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:16.463075 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:16.463256 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:16.463537 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.464649 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.466149 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.466904 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.467058 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:16.467254 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:16.467454 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 17:14:16.467566 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":64,"threads_started":1}
I20260811 17:14:16.467728 27077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:16.460909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:33439 (local address 127.0.0.1:54552)
0811 17:14:16.461350 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:16.461400 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:16.461529 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:16.462432 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:16.462442 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:16.462475 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:16.462648 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:16.462658 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.464221 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.464225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:16.465991 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:16.466003 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.467068 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.467071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:16.467217 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:16.467256 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 17:14:16.467361 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 17:14:16.467536 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":64,"thread_start_us":115,"threads_started":1}
I20260811 17:14:16.468039 27079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:16.460919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:38671 (local address 127.0.0.1:43648)
0811 17:14:16.462555 (+ 1636us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:16.462574 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:16.462678 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:16.463419 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:16.463425 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:16.463448 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:16.463547 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:16.463552 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.465000 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.465004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:16.466320 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:16.466332 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.467645 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.467648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:16.467763 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:16.467786 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 17:14:16.467851 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 17:14:16.467921 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1565,"mutex_wait_us":20,"spinlock_wait_cycles":15616,"thread_start_us":144,"threads_started":1}
I20260811 17:14:16.468494 27081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:16.462160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43648 (local address 127.21.54.130:38671)
0811 17:14:16.462771 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:16.462784 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:16.462805 (+ 21us) server_negotiation.cc:407] Connection header received
0811 17:14:16.462881 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:16.462885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:16.463066 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:16.463254 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:16.463687 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.464826 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.466514 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.467334 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.467433 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:16.467570 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:16.468237 (+ 667us) server_negotiation.cc:299] Negotiation successful
0811 17:14:16.468347 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":273,"threads_started":1}
I20260811 17:14:16.468639 27078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:16.461060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54552 (local address 127.21.54.131:33439)
0811 17:14:16.461813 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:16.461828 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:16.461853 (+ 25us) server_negotiation.cc:407] Connection header received
0811 17:14:16.461937 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:16.461942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:16.462218 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:16.462403 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:16.462805 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.464034 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.466151 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.466920 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.467802 (+ 882us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:16.468291 (+ 489us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:16.468346 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 17:14:16.468442 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":168,"threads_started":1}
I20260811 17:14:16.467723 27076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:16.461724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:44835 (local address 127.0.0.1:56898)
0811 17:14:16.462688 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:16.462703 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:16.462791 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:16.463231 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:16.463235 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:16.463254 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:16.463388 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:16.463397 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.464790 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.464795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:16.465991 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:16.466003 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.467064 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.467068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:16.467205 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:16.467256 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 17:14:16.467418 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 17:14:16.467547 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"mutex_wait_us":100,"thread_start_us":95,"threads_started":1}
I20260811 17:14:16.471741 26990 tablet_service.cc:1467] Processing CreateTablet for tablet dbd3e4926ff3460286d226ca46db7656 (DEFAULT_TABLE table=test-table [id=50a20b992ecf4263a434d514bbe1bac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:16.471738 26676 tablet_service.cc:1467] Processing CreateTablet for tablet dbd3e4926ff3460286d226ca46db7656 (DEFAULT_TABLE table=test-table [id=50a20b992ecf4263a434d514bbe1bac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:16.472380 26831 tablet_service.cc:1467] Processing CreateTablet for tablet dbd3e4926ff3460286d226ca46db7656 (DEFAULT_TABLE table=test-table [id=50a20b992ecf4263a434d514bbe1bac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:16.473007 26990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd3e4926ff3460286d226ca46db7656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:16.473002 26676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd3e4926ff3460286d226ca46db7656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:16.473392 26831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd3e4926ff3460286d226ca46db7656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:16.480352 27084 tablet_bootstrap.cc:492] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6: Bootstrap starting.
I20260811 17:14:16.481156 27083 tablet_bootstrap.cc:492] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5: Bootstrap starting.
I20260811 17:14:16.481248 27082 tablet_bootstrap.cc:492] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7: Bootstrap starting.
I20260811 17:14:16.484112 27082 tablet_bootstrap.cc:654] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:16.484112 27083 tablet_bootstrap.cc:654] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:16.485312 27083 log.cc:824] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:16.485532 27082 log.cc:824] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:16.487133 27084 tablet_bootstrap.cc:654] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:16.487903 27083 tablet_bootstrap.cc:492] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5: No bootstrap required, opened a new log
I20260811 17:14:16.488163 27083 ts_tablet_manager.cc:1397] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 17:14:16.488725 27084 log.cc:824] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:16.489012 27082 tablet_bootstrap.cc:492] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7: No bootstrap required, opened a new log
I20260811 17:14:16.489289 27082 ts_tablet_manager.cc:1397] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 17:14:16.491015 27084 tablet_bootstrap.cc:492] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6: No bootstrap required, opened a new log
I20260811 17:14:16.491291 27084 ts_tablet_manager.cc:1397] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260811 17:14:16.492995 27083 raft_consensus.cc:348] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a9776decd6e49a181c138d675f0b1c7" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 33439 } } peers { permanent_uuid: "7cf9ba13b2df4d8781b21c44f4185bd6" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38671 } } peers { permanent_uuid: "03e1d079e0734d7bbb8a048e25d9dca5" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44835 } }
I20260811 17:14:16.493296 27083 raft_consensus.cc:374] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:16.493389 27083 raft_consensus.cc:729] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03e1d079e0734d7bbb8a048e25d9dca5, State: Initialized, Role: FOLLOWER
I20260811 17:14:16.493644 27082 raft_consensus.cc:348] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a9776decd6e49a181c138d675f0b1c7" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 33439 } } peers { permanent_uuid: "7cf9ba13b2df4d8781b21c44f4185bd6" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38671 } } peers { permanent_uuid: "03e1d079e0734d7bbb8a048e25d9dca5" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44835 } }
I20260811 17:14:16.493932 27082 raft_consensus.cc:374] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:16.494052 27082 raft_consensus.cc:729] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a9776decd6e49a181c138d675f0b1c7, State: Initialized, Role: FOLLOWER
I20260811 17:14:16.493983 27083 consensus_queue.cc:260] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a9776decd6e49a181c138d675f0b1c7" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 33439 } } peers { permanent_uuid: "7cf9ba13b2df4d8781b21c44f4185bd6" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38671 } } peers { permanent_uuid: "03e1d079e0734d7bbb8a048e25d9dca5" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44835 } }
I20260811 17:14:16.494798 26756 heartbeater.cc:503] Master 127.21.54.190:43761 was elected leader, sending a full tablet report...
I20260811 17:14:16.495184 27083 ts_tablet_manager.cc:1428] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 17:14:16.494530 27082 consensus_queue.cc:260] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a9776decd6e49a181c138d675f0b1c7" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 33439 } } peers { permanent_uuid: "7cf9ba13b2df4d8781b21c44f4185bd6" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38671 } } peers { permanent_uuid: "03e1d079e0734d7bbb8a048e25d9dca5" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44835 } }
I20260811 17:14:16.495720 27084 raft_consensus.cc:348] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a9776decd6e49a181c138d675f0b1c7" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 33439 } } peers { permanent_uuid: "7cf9ba13b2df4d8781b21c44f4185bd6" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38671 } } peers { permanent_uuid: "03e1d079e0734d7bbb8a048e25d9dca5" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44835 } }
I20260811 17:14:16.496091 27084 raft_consensus.cc:374] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:16.496089 27065 heartbeater.cc:503] Master 127.21.54.190:43761 was elected leader, sending a full tablet report...
I20260811 17:14:16.496208 27084 raft_consensus.cc:729] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cf9ba13b2df4d8781b21c44f4185bd6, State: Initialized, Role: FOLLOWER
I20260811 17:14:16.496452 27082 ts_tablet_manager.cc:1428] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 17:14:16.496807 27084 consensus_queue.cc:260] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a9776decd6e49a181c138d675f0b1c7" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 33439 } } peers { permanent_uuid: "7cf9ba13b2df4d8781b21c44f4185bd6" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38671 } } peers { permanent_uuid: "03e1d079e0734d7bbb8a048e25d9dca5" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44835 } }
I20260811 17:14:16.497715 26912 heartbeater.cc:503] Master 127.21.54.190:43761 was elected leader, sending a full tablet report...
I20260811 17:14:16.498059 27084 ts_tablet_manager.cc:1428] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 17:14:16.564620 27089 raft_consensus.cc:482] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:14:16.565101 27089 raft_consensus.cc:504] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a9776decd6e49a181c138d675f0b1c7" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 33439 } } peers { permanent_uuid: "7cf9ba13b2df4d8781b21c44f4185bd6" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38671 } } peers { permanent_uuid: "03e1d079e0734d7bbb8a048e25d9dca5" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44835 } }
I20260811 17:14:16.566632 27089 leader_election.cc:288] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a9776decd6e49a181c138d675f0b1c7 (127.21.54.131:33439), 7cf9ba13b2df4d8781b21c44f4185bd6 (127.21.54.130:38671)
I20260811 17:14:16.571940 27078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:16.566988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:46875 (local address 127.21.54.131:33439)
0811 17:14:16.567119 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:16.567124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:16.567583 (+ 459us) server_negotiation.cc:407] Connection header received
0811 17:14:16.567772 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:16.567775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:16.567825 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:16.567914 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:16.568462 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.569118 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.570627 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.571159 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.571342 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:16.571426 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:16.571689 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 17:14:16.571784 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 17:14:16.571976 27093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:16.566938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:33439 (local address 127.21.54.129:46875)
0811 17:14:16.567459 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:16.567475 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:16.567652 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:16.567987 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:16.567989 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:16.568149 (+ 160us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:16.568296 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:16.568303 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.569266 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.569269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:16.570441 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:16.570455 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.571341 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.571346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:16.571494 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:16.571520 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 17:14:16.571660 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 17:14:16.571760 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":157,"thread_start_us":138,"threads_started":1}
I20260811 17:14:16.572011 27081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:16.567033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:42267 (local address 127.21.54.130:38671)
0811 17:14:16.567187 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:16.567192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:16.567458 (+ 266us) server_negotiation.cc:407] Connection header received
0811 17:14:16.567574 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:16.567577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:16.567627 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:16.567712 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:16.568212 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.568889 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.570652 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.571241 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.571373 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:16.571440 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:16.571799 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 17:14:16.571883 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 17:14:16.572448 27092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:16.566938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:38671 (local address 127.21.54.129:42267)
0811 17:14:16.567324 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:16.567340 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:16.567477 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:16.567756 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:16.567759 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:16.567778 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:16.567952 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:16.567960 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.569012 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.569015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:16.570453 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:16.570466 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.571522 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.571528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:16.571655 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:16.571680 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 17:14:16.572226 (+ 546us) client_negotiation.cc:240] Negotiation successful
0811 17:14:16.572304 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":1,"thread_start_us":155,"threads_started":1}
I20260811 17:14:16.572918 27011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd3e4926ff3460286d226ca46db7656" candidate_uuid: "03e1d079e0734d7bbb8a048e25d9dca5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a9776decd6e49a181c138d675f0b1c7" is_pre_election: true
I20260811 17:14:16.573200 26856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd3e4926ff3460286d226ca46db7656" candidate_uuid: "03e1d079e0734d7bbb8a048e25d9dca5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cf9ba13b2df4d8781b21c44f4185bd6" is_pre_election: true
I20260811 17:14:16.573304 27011 raft_consensus.cc:2454] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03e1d079e0734d7bbb8a048e25d9dca5 in term 0.
I20260811 17:14:16.573524 26856 raft_consensus.cc:2454] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03e1d079e0734d7bbb8a048e25d9dca5 in term 0.
I20260811 17:14:16.573953 26622 leader_election.cc:302] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03e1d079e0734d7bbb8a048e25d9dca5, 4a9776decd6e49a181c138d675f0b1c7; no voters:
I20260811 17:14:16.574322 27089 raft_consensus.cc:2781] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:14:16.574432 27089 raft_consensus.cc:482] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:14:16.574479 27089 raft_consensus.cc:3037] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:16.575850 27089 raft_consensus.cc:504] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a9776decd6e49a181c138d675f0b1c7" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 33439 } } peers { permanent_uuid: "7cf9ba13b2df4d8781b21c44f4185bd6" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38671 } } peers { permanent_uuid: "03e1d079e0734d7bbb8a048e25d9dca5" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44835 } }
I20260811 17:14:16.576289 27089 leader_election.cc:288] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 [CANDIDATE]: Term 1 election: Requested vote from peers 4a9776decd6e49a181c138d675f0b1c7 (127.21.54.131:33439), 7cf9ba13b2df4d8781b21c44f4185bd6 (127.21.54.130:38671)
I20260811 17:14:16.576889 27011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd3e4926ff3460286d226ca46db7656" candidate_uuid: "03e1d079e0734d7bbb8a048e25d9dca5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a9776decd6e49a181c138d675f0b1c7"
I20260811 17:14:16.576890 26856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd3e4926ff3460286d226ca46db7656" candidate_uuid: "03e1d079e0734d7bbb8a048e25d9dca5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cf9ba13b2df4d8781b21c44f4185bd6"
I20260811 17:14:16.577057 26856 raft_consensus.cc:3037] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:16.577064 27011 raft_consensus.cc:3037] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:16.578302 27011 raft_consensus.cc:2454] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03e1d079e0734d7bbb8a048e25d9dca5 in term 1.
I20260811 17:14:16.578459 26856 raft_consensus.cc:2454] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03e1d079e0734d7bbb8a048e25d9dca5 in term 1.
I20260811 17:14:16.578730 26622 leader_election.cc:302] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03e1d079e0734d7bbb8a048e25d9dca5, 4a9776decd6e49a181c138d675f0b1c7; no voters:
I20260811 17:14:16.578977 27089 raft_consensus.cc:2781] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:16.579234 27089 raft_consensus.cc:686] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 [term 1 LEADER]: Becoming Leader. State: Replica: 03e1d079e0734d7bbb8a048e25d9dca5, State: Running, Role: LEADER
I20260811 17:14:16.579605 27089 consensus_queue.cc:237] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a9776decd6e49a181c138d675f0b1c7" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 33439 } } peers { permanent_uuid: "7cf9ba13b2df4d8781b21c44f4185bd6" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38671 } } peers { permanent_uuid: "03e1d079e0734d7bbb8a048e25d9dca5" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44835 } }
I20260811 17:14:16.583312 26525 catalog_manager.cc:5495] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03e1d079e0734d7bbb8a048e25d9dca5 (127.21.54.129). New cstate: current_term: 1 leader_uuid: "03e1d079e0734d7bbb8a048e25d9dca5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a9776decd6e49a181c138d675f0b1c7" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 33439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cf9ba13b2df4d8781b21c44f4185bd6" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03e1d079e0734d7bbb8a048e25d9dca5" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44835 } health_report { overall_health: HEALTHY } } }
I20260811 17:14:16.626591 27080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:16.622218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56902 (local address 127.21.54.129:44835)
0811 17:14:16.622395 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:16.622403 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:16.622422 (+ 19us) server_negotiation.cc:407] Connection header received
0811 17:14:16.622586 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:16.622588 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:16.622650 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:16.622781 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:14:16.623092 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.623694 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.624718 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:16.624955 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:16.625078 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:16.625465 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:14:16.626106 (+ 641us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:14:16.626293 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:16.626334 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 17:14:16.626405 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 17:14:16.627818 26757 tablet.cc:2369] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:14:16.633967 26856 raft_consensus.cc:1261] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6 [term 1 FOLLOWER]: Refusing update from remote peer 03e1d079e0734d7bbb8a048e25d9dca5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:16.633968 27011 raft_consensus.cc:1261] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7 [term 1 FOLLOWER]: Refusing update from remote peer 03e1d079e0734d7bbb8a048e25d9dca5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:16.634868 27094 consensus_queue.cc:1035] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a9776decd6e49a181c138d675f0b1c7" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 33439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:16.635166 27089 consensus_queue.cc:1035] T dbd3e4926ff3460286d226ca46db7656 P 03e1d079e0734d7bbb8a048e25d9dca5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cf9ba13b2df4d8781b21c44f4185bd6" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 17:14:16.646549 26913 tablet.cc:2369] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:14:16.647969 27099 mvcc.cc:205] Tried to move back new op lower bound from 7317374798364164096 to 7317374798153191424. Current Snapshot: MvccSnapshot[applied={T|T < 7317374798364164096}]
I20260811 17:14:16.649778 27100 mvcc.cc:205] Tried to move back new op lower bound from 7317374798364164096 to 7317374798153191424. Current Snapshot: MvccSnapshot[applied={T|T < 7317374798364164096}]
I20260811 17:14:16.656561 26521 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:16.656808 26521 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:16.660324 26521 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 78377c4aee544fd78cb67021497c544b: Sending DeleteTablet for 3 replicas of tablet dbd3e4926ff3460286d226ca46db7656
I20260811 17:14:16.661558 26990 tablet_service.cc:1514] Processing DeleteTablet for tablet dbd3e4926ff3460286d226ca46db7656 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:16 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:54552
I20260811 17:14:16.661540 26676 tablet_service.cc:1514] Processing DeleteTablet for tablet dbd3e4926ff3460286d226ca46db7656 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:16 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:56898
I20260811 17:14:16.662278 26831 tablet_service.cc:1514] Processing DeleteTablet for tablet dbd3e4926ff3460286d226ca46db7656 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:16 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:43648
I20260811 17:14:16.662770 27109 tablet_replica.cc:330] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7: stopping tablet replica
I20260811 17:14:16.663053 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 26603
I20260811 17:14:16.663244 27109 raft_consensus.cc:2229] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:16.663275 27111 tablet_replica.cc:330] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6: stopping tablet replica
I20260811 17:14:16.663640 27111 raft_consensus.cc:2229] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:16.663730 27109 raft_consensus.cc:2258] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:16.663976 27111 raft_consensus.cc:2258] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:16.667083 27111 ts_tablet_manager.cc:1905] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:16.671610 27111 ts_tablet_manager.cc:1918] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:14:16.671769 27111 log.cc:1196] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/dbd3e4926ff3460286d226ca46db7656
I20260811 17:14:16.672151 27111 ts_tablet_manager.cc:1939] T dbd3e4926ff3460286d226ca46db7656 P 7cf9ba13b2df4d8781b21c44f4185bd6: Deleting consensus metadata
I20260811 17:14:16.673221 27109 ts_tablet_manager.cc:1905] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:16.674338 26510 catalog_manager.cc:4841] TS 7cf9ba13b2df4d8781b21c44f4185bd6 (127.21.54.130:38671): tablet dbd3e4926ff3460286d226ca46db7656 (table test-table [id=50a20b992ecf4263a434d514bbe1bac4]) successfully deleted
I20260811 17:14:16.675997 27109 ts_tablet_manager.cc:1918] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:14:16.676306 27109 log.cc:1196] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/dbd3e4926ff3460286d226ca46db7656
I20260811 17:14:16.676764 27109 ts_tablet_manager.cc:1939] T dbd3e4926ff3460286d226ca46db7656 P 4a9776decd6e49a181c138d675f0b1c7: Deleting consensus metadata
I20260811 17:14:16.678223 26512 catalog_manager.cc:4841] TS 4a9776decd6e49a181c138d675f0b1c7 (127.21.54.131:33439): tablet dbd3e4926ff3460286d226ca46db7656 (table test-table [id=50a20b992ecf4263a434d514bbe1bac4]) successfully deleted
I20260811 17:14:16.679311 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 26761
W20260811 17:14:16.679304 26510 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.54.129:44835 (error 108)
I20260811 17:14:16.680826 27076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:16.680352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:44835 (local address 127.0.0.1:56918)
0811 17:14:16.680640 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:16.680715 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.129:44835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
W20260811 17:14:16.681056 26510 catalog_manager.cc:4584] TS 03e1d079e0734d7bbb8a048e25d9dca5 (127.21.54.129:44835): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dbd3e4926ff3460286d226ca46db7656: Network error: Client connection negotiation failed: client connection to 127.21.54.129:44835: connect: Connection refused (error 111)
I20260811 17:14:16.689934 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 26919
I20260811 17:14:16.700243 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 26493
2026-08-11T17:14:16Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (3972 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T17:14:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:14:16Z Disabled control of system clock
I20260811 17:14:16.741087 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:36443
--webserver_interface=127.21.54.190
--webserver_port=0
--builtin_ntp_servers=127.21.54.148:40347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:36443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 17:14:16.865262 27117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:16.865624 27117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:16.865666 27117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:16.870115 27117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:16.870203 27117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:16.870219 27117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:16.870237 27117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:16.870253 27117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:14:16.875484 27117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:40347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:36443
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:36443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:16.876753 27117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:16.878332 27117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:16.888087 27125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:16.888098 27122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:16.888060 27128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:16.888614 27117 server_base.cc:1034] running on GCE node
I20260811 17:14:16.889339 27117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:16.890717 27117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:16.891964 27117 hybrid_clock.cc:648] HybridClock initialized: now 1786468456891932 us; error 49 us; skew 500 ppm
I20260811 17:14:16.894373 27117 webserver.cc:460] Webserver started at http://127.21.54.190:46129/ using document root <none> and password file <none>
I20260811 17:14:16.895066 27117 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:16.895174 27117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:16.895437 27117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:16.897878 27117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "1fa813fb48654c5d9771528e344fe89f"
format_stamp: "Formatted at 2026-08-11 17:14:16 on dist-test-slave-fd7l"
server_key: "347a930b18ff23ed895732bf64a74982"
server_key_iv: "77049856a1e250d60dc749a7c524e804"
server_key_version: "encryptionkey@0"
I20260811 17:14:16.898581 27117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "1fa813fb48654c5d9771528e344fe89f"
format_stamp: "Formatted at 2026-08-11 17:14:16 on dist-test-slave-fd7l"
server_key: "347a930b18ff23ed895732bf64a74982"
server_key_iv: "77049856a1e250d60dc749a7c524e804"
server_key_version: "encryptionkey@0"
I20260811 17:14:16.903066 27117 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:14:16.906013 27135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:16.907711 27117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:14:16.907907 27117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "1fa813fb48654c5d9771528e344fe89f"
format_stamp: "Formatted at 2026-08-11 17:14:16 on dist-test-slave-fd7l"
server_key: "347a930b18ff23ed895732bf64a74982"
server_key_iv: "77049856a1e250d60dc749a7c524e804"
server_key_version: "encryptionkey@0"
I20260811 17:14:16.908072 27117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:16.913292 27117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:16.914016 27117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:16.914245 27117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:16.922060 27117 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:36443
I20260811 17:14:16.922135 27196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:36443 every 8 connection(s)
I20260811 17:14:16.923168 27117 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:14:16.927223 27197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:16.929361 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 27117
I20260811 17:14:16.929513 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:14:16.929883 21722 external_mini_cluster.cc:1428] Setting key 1e50b92132d509c7a37d18954e8d63a8
I20260811 17:14:16.935578 27197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f: Bootstrap starting.
I20260811 17:14:16.936688 27202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:16.931756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42698 (local address 127.21.54.190:36443)
0811 17:14:16.932642 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:16.932666 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:16.932706 (+ 40us) server_negotiation.cc:407] Connection header received
0811 17:14:16.933421 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:16.933442 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:16.933830 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:16.934253 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:16.935215 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:16.935220 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:16.935234 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:16.935245 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:16.935306 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:16.935323 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:16.935492 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:16.935569 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 17:14:16.935847 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":170,"threads_started":1}
I20260811 17:14:16.938426 27197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:16.939432 27197 log.cc:824] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:16.942317 27197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f: No bootstrap required, opened a new log
I20260811 17:14:16.947283 27197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fa813fb48654c5d9771528e344fe89f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 36443 } }
I20260811 17:14:16.947682 27197 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:16.947783 27197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fa813fb48654c5d9771528e344fe89f, State: Initialized, Role: FOLLOWER
I20260811 17:14:16.948401 27197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fa813fb48654c5d9771528e344fe89f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 36443 } }
I20260811 17:14:16.948621 27197 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:14:16.948730 27197 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:14:16.948855 27197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:16.950075 27197 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fa813fb48654c5d9771528e344fe89f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 36443 } }
I20260811 17:14:16.950528 27197 leader_election.cc:302] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fa813fb48654c5d9771528e344fe89f; no voters:
I20260811 17:14:16.950958 27197 leader_election.cc:288] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:14:16.951159 27205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:16.951442 27205 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f [term 1 LEADER]: Becoming Leader. State: Replica: 1fa813fb48654c5d9771528e344fe89f, State: Running, Role: LEADER
I20260811 17:14:16.951846 27205 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fa813fb48654c5d9771528e344fe89f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 36443 } }
I20260811 17:14:16.952450 27197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:16.953852 27206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fa813fb48654c5d9771528e344fe89f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fa813fb48654c5d9771528e344fe89f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 36443 } } }
I20260811 17:14:16.953835 27207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fa813fb48654c5d9771528e344fe89f. Latest consensus state: current_term: 1 leader_uuid: "1fa813fb48654c5d9771528e344fe89f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fa813fb48654c5d9771528e344fe89f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 36443 } } }
I20260811 17:14:16.954121 27206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:16.954121 27207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:16.954675 27214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:14:16.958148 27214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:14:16.964293 27214 catalog_manager.cc:1347] Generated new cluster ID: 4637f249361b406fa9c0745f2ddf5a5a
I20260811 17:14:16.964393 27214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:14:16.982023 27214 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:14:16.982482 27214 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:14:16.994323 27214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f: Generated new TSK 0
I20260811 17:14:16.995275 27214 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:14:16.998647 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:36443
--builtin_ntp_servers=127.21.54.148:40347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 17:14:17.145831 27225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:17.146255 27225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:17.146301 27225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:17.150908 27225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:17.150990 27225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:17.151053 27225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:14:17.156762 27225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:40347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:36443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:17.157984 27225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:17.159484 27225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:17.170456 27232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:17.170478 27234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:17.170516 27231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:17.171006 27225 server_base.cc:1034] running on GCE node
I20260811 17:14:17.171507 27225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:17.172294 27225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:17.173532 27225 hybrid_clock.cc:648] HybridClock initialized: now 1786468457173505 us; error 42 us; skew 500 ppm
I20260811 17:14:17.176009 27225 webserver.cc:460] Webserver started at http://127.21.54.129:42385/ using document root <none> and password file <none>
I20260811 17:14:17.176862 27225 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:17.176975 27225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:17.177254 27225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:17.179687 27225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "a1057775631e450abecbfd5516db846f"
format_stamp: "Formatted at 2026-08-11 17:14:17 on dist-test-slave-fd7l"
server_key: "d185dab48bcde870cd54a88662eb6bc8"
server_key_iv: "8f0b24f4f24ea905fcef9e148896b204"
server_key_version: "encryptionkey@0"
I20260811 17:14:17.180336 27225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "a1057775631e450abecbfd5516db846f"
format_stamp: "Formatted at 2026-08-11 17:14:17 on dist-test-slave-fd7l"
server_key: "d185dab48bcde870cd54a88662eb6bc8"
server_key_iv: "8f0b24f4f24ea905fcef9e148896b204"
server_key_version: "encryptionkey@0"
I20260811 17:14:17.184908 27225 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:14:17.188006 27241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:17.189754 27225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:17.189949 27225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "a1057775631e450abecbfd5516db846f"
format_stamp: "Formatted at 2026-08-11 17:14:17 on dist-test-slave-fd7l"
server_key: "d185dab48bcde870cd54a88662eb6bc8"
server_key_iv: "8f0b24f4f24ea905fcef9e148896b204"
server_key_version: "encryptionkey@0"
I20260811 17:14:17.190105 27225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:17.195629 27225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:17.196432 27225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:17.196708 27225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:17.197384 27225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:17.198392 27225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:17.198441 27225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:17.198480 27225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:17.198495 27225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:17.208071 27225 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:33877
I20260811 17:14:17.208165 27374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:33877 every 8 connection(s)
I20260811 17:14:17.209182 27225 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:14:17.216904 27202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:17.211328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:57953 (local address 127.21.54.190:36443)
0811 17:14:17.211493 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:17.211498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:17.213170 (+ 1672us) server_negotiation.cc:407] Connection header received
0811 17:14:17.214662 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:17.214670 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:17.214717 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:17.214804 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:17.216186 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:17.216189 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:17.216191 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:17.216207 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:17.216232 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:17.216241 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:17.216342 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:17.216711 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 17:14:17.216769 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 17:14:17.217468 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 27225
I20260811 17:14:17.217617 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:14:17.217958 21722 external_mini_cluster.cc:1428] Setting key fbaff09ea1e7c25ae77e82ac48c141e2
I20260811 17:14:17.218217 27378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:17.211847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:36443 (local address 127.21.54.129:57953)
0811 17:14:17.212955 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:17.213033 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:17.214340 (+ 1307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:17.215050 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:17.215058 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:17.215165 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:17.215857 (+ 692us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:14:17.215859 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:17.215907 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:14:17.215991 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:17.216368 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:17.216380 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:17.216386 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 17:14:17.216759 (+ 373us) client_negotiation.cc:240] Negotiation successful
0811 17:14:17.217072 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":153,"threads_started":1}
I20260811 17:14:17.219949 27375 heartbeater.cc:347] Connected to a master server at 127.21.54.190:36443
I20260811 17:14:17.220254 27375 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:17.220635 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:36443
--builtin_ntp_servers=127.21.54.148:40347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 17:14:17.220841 27375 heartbeater.cc:511] Master 127.21.54.190:36443 requested a full tablet report, sending...
I20260811 17:14:17.222913 27153 ts_manager.cc:194] Registered new tserver with Master: a1057775631e450abecbfd5516db846f (127.21.54.129:33877)
W20260811 17:14:17.346150 27379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:17.346626 27379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:17.346678 27379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:17.351703 27379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:17.351792 27379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:17.351856 27379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:14:17.357543 27379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:40347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:36443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:17.358906 27379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:17.360389 27379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:17.371238 27386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:17.371219 27388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:17.371336 27379 server_base.cc:1034] running on GCE node
W20260811 17:14:17.371219 27385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:17.372099 27379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:17.372922 27379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:17.374145 27379 hybrid_clock.cc:648] HybridClock initialized: now 1786468457374111 us; error 50 us; skew 500 ppm
I20260811 17:14:17.376472 27379 webserver.cc:460] Webserver started at http://127.21.54.130:42657/ using document root <none> and password file <none>
I20260811 17:14:17.377203 27379 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:17.377321 27379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:17.377583 27379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:17.380007 27379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "89a1211cc646483881d9f2993b141a1f"
format_stamp: "Formatted at 2026-08-11 17:14:17 on dist-test-slave-fd7l"
server_key: "b2342d21f0b724fc6c6b70cfa1cdafe0"
server_key_iv: "3cdff16dbcba743121de2c2003cdcc02"
server_key_version: "encryptionkey@0"
I20260811 17:14:17.380718 27379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "89a1211cc646483881d9f2993b141a1f"
format_stamp: "Formatted at 2026-08-11 17:14:17 on dist-test-slave-fd7l"
server_key: "b2342d21f0b724fc6c6b70cfa1cdafe0"
server_key_iv: "3cdff16dbcba743121de2c2003cdcc02"
server_key_version: "encryptionkey@0"
I20260811 17:14:17.385094 27379 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:14:17.388000 27395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:17.389778 27379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:17.389955 27379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "89a1211cc646483881d9f2993b141a1f"
format_stamp: "Formatted at 2026-08-11 17:14:17 on dist-test-slave-fd7l"
server_key: "b2342d21f0b724fc6c6b70cfa1cdafe0"
server_key_iv: "3cdff16dbcba743121de2c2003cdcc02"
server_key_version: "encryptionkey@0"
I20260811 17:14:17.390100 27379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:17.395377 27379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:17.396111 27379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:17.396332 27379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:17.397058 27379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:17.398135 27379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:17.398213 27379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:17.398283 27379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:17.398321 27379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:17.408214 27379 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:42125
I20260811 17:14:17.408233 27523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:42125 every 8 connection(s)
I20260811 17:14:17.409333 27379 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:14:17.409564 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 27379
I20260811 17:14:17.409694 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:14:17.410059 21722 external_mini_cluster.cc:1428] Setting key 981e070bda9d0ed646415ae58be785ca
I20260811 17:14:17.412586 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:36443
--builtin_ntp_servers=127.21.54.148:40347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 17:14:17.416987 27202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:17.411637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:60827 (local address 127.21.54.190:36443)
0811 17:14:17.411778 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:17.411783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:17.413177 (+ 1394us) server_negotiation.cc:407] Connection header received
0811 17:14:17.414825 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:17.414828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:17.414869 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:17.414950 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:17.416296 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:17.416303 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:17.416309 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:17.416326 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:17.416353 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:17.416366 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:17.416612 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:17.416825 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 17:14:17.416869 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 17:14:17.418310 27526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:17.412134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:36443 (local address 127.21.54.130:60827)
0811 17:14:17.413019 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:17.413076 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:17.414546 (+ 1470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:17.415150 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:17.415162 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:17.415291 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:17.415962 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:14:17.415964 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:17.416007 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:14:17.416111 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:17.416660 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:17.416673 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:17.416683 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 17:14:17.416871 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 17:14:17.417130 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":153,"threads_started":1}
I20260811 17:14:17.419711 27524 heartbeater.cc:347] Connected to a master server at 127.21.54.190:36443
I20260811 17:14:17.419987 27524 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:17.420457 27524 heartbeater.cc:511] Master 127.21.54.190:36443 requested a full tablet report, sending...
I20260811 17:14:17.421734 27153 ts_manager.cc:194] Registered new tserver with Master: 89a1211cc646483881d9f2993b141a1f (127.21.54.130:42125)
W20260811 17:14:17.541997 27527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:17.542317 27527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:17.542362 27527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:17.546761 27527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:17.546867 27527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:17.546941 27527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:14:17.553169 27527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:40347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:36443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:17.554580 27527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:17.556128 27527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:17.567836 27533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:17.567792 27535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:17.567837 27532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:17.568156 27527 server_base.cc:1034] running on GCE node
I20260811 17:14:17.568848 27527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:17.569705 27527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:17.570942 27527 hybrid_clock.cc:648] HybridClock initialized: now 1786468457570935 us; error 48 us; skew 500 ppm
I20260811 17:14:17.573395 27527 webserver.cc:460] Webserver started at http://127.21.54.131:35653/ using document root <none> and password file <none>
I20260811 17:14:17.574110 27527 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:17.574224 27527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:17.574488 27527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:17.576941 27527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "58fb3a02b17446d6a6219c40b4ac1df9"
format_stamp: "Formatted at 2026-08-11 17:14:17 on dist-test-slave-fd7l"
server_key: "b4b998ec786a0824eecc95acfe6c9ff6"
server_key_iv: "7a9f30dc80a0adbaffbcdbdf36c6b62e"
server_key_version: "encryptionkey@0"
I20260811 17:14:17.577592 27527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "58fb3a02b17446d6a6219c40b4ac1df9"
format_stamp: "Formatted at 2026-08-11 17:14:17 on dist-test-slave-fd7l"
server_key: "b4b998ec786a0824eecc95acfe6c9ff6"
server_key_iv: "7a9f30dc80a0adbaffbcdbdf36c6b62e"
server_key_version: "encryptionkey@0"
I20260811 17:14:17.581915 27527 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 17:14:17.584947 27541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:17.586459 27527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:14:17.586690 27527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "58fb3a02b17446d6a6219c40b4ac1df9"
format_stamp: "Formatted at 2026-08-11 17:14:17 on dist-test-slave-fd7l"
server_key: "b4b998ec786a0824eecc95acfe6c9ff6"
server_key_iv: "7a9f30dc80a0adbaffbcdbdf36c6b62e"
server_key_version: "encryptionkey@0"
I20260811 17:14:17.586843 27527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:17.592140 27527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:17.592908 27527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:17.593137 27527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:17.593848 27527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:17.594985 27527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:17.595062 27527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:17.595129 27527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:17.595177 27527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:17.605122 27527 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:38471
I20260811 17:14:17.605195 27675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:38471 every 8 connection(s)
I20260811 17:14:17.606207 27527 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:14:17.612156 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 27527
I20260811 17:14:17.612310 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:14:17.612718 21722 external_mini_cluster.cc:1428] Setting key 9e93b2c65240220ec4e6bf86d446b5dc
I20260811 17:14:17.613583 27202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:17.608435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:33157 (local address 127.21.54.190:36443)
0811 17:14:17.608678 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:17.608682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:17.609915 (+ 1233us) server_negotiation.cc:407] Connection header received
0811 17:14:17.611256 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:17.611259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:17.611300 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:17.611383 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:17.613026 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:17.613031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:17.613033 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:17.613052 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:17.613075 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:17.613091 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:17.613189 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:17.613382 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 17:14:17.613432 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260811 17:14:17.614639 27678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:17.608965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:36443 (local address 127.21.54.131:33157)
0811 17:14:17.609746 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:17.609790 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:17.611030 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:17.611635 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:17.611642 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:17.611735 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:17.612603 (+ 868us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:14:17.612605 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:17.612697 (+ 92us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:14:17.612803 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:17.613208 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:17.613220 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:17.613227 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 17:14:17.613405 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 17:14:17.613699 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":125,"threads_started":1}
I20260811 17:14:17.615835 27676 heartbeater.cc:347] Connected to a master server at 127.21.54.190:36443
I20260811 17:14:17.616061 27676 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:17.616806 27676 heartbeater.cc:511] Master 127.21.54.190:36443 requested a full tablet report, sending...
I20260811 17:14:17.618014 27153 ts_manager.cc:194] Registered new tserver with Master: 58fb3a02b17446d6a6219c40b4ac1df9 (127.21.54.131:38471)
I20260811 17:14:17.627710 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:14:17.632138 27202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:17.629995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42706 (local address 127.21.54.190:36443)
0811 17:14:17.630151 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:17.630156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:17.630367 (+ 211us) server_negotiation.cc:407] Connection header received
0811 17:14:17.630497 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:17.630500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:17.630545 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:17.630622 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:17.631497 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:17.631501 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:17.631504 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:17.631518 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:17.631541 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:17.631550 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:17.631638 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:17.631848 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 17:14:17.631951 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 17:14:17.635792 27153 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:14:17.639161 27153 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:14:17.652942 27691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:17.649897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47956 (local address 127.21.54.131:38471)
0811 17:14:17.650412 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:17.650422 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:17.650441 (+ 19us) server_negotiation.cc:407] Connection header received
0811 17:14:17.650486 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:17.650490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:17.650621 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:17.650917 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:17.652337 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:17.652343 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:17.652353 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:17.652367 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:17.652410 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:17.652443 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:17.652576 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:17.652697 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 17:14:17.652778 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":102,"threads_started":1}
I20260811 17:14:17.652925 27688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:17.649414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:33877 (local address 127.0.0.1:43086)
0811 17:14:17.649910 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:17.649962 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:17.650076 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:17.650869 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:17.650877 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:17.650955 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:17.651575 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:14:17.651577 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:17.651598 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:14:17.651690 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:17.652456 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:17.652469 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:17.652478 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 17:14:17.652619 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 17:14:17.652749 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":19,"thread_start_us":103,"threads_started":1}
I20260811 17:14:17.652930 27689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:17.649498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:38471 (local address 127.0.0.1:47956)
0811 17:14:17.650295 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:17.650311 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:17.650387 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:17.651625 (+ 1238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:17.651628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:17.651642 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:17.652113 (+ 471us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:14:17.652115 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:17.652127 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:14:17.652155 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:17.652542 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:17.652545 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:17.652549 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 17:14:17.652648 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 17:14:17.652732 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":652,"threads_started":1}
I20260811 17:14:17.653605 27692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:17.650244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:42125 (local address 127.0.0.1:59720)
0811 17:14:17.651034 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:17.651048 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:17.651118 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:17.651843 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:17.651847 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:17.651864 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:17.652255 (+ 391us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:14:17.652257 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:17.652268 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:14:17.652295 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:17.653396 (+ 1101us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:17.653399 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:17.653403 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 17:14:17.653459 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 17:14:17.653493 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":95,"threads_started":1}
I20260811 17:14:17.653627 27690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:17.649516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43086 (local address 127.21.54.129:33877)
0811 17:14:17.650311 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:17.650324 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:17.650352 (+ 28us) server_negotiation.cc:407] Connection header received
0811 17:14:17.650410 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:17.650417 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:17.650630 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:17.651263 (+ 633us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:17.652240 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:17.652248 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:17.652263 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:17.652276 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:17.652321 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:17.652350 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:17.653370 (+ 1020us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:17.653421 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 17:14:17.653489 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":187,"threads_started":1}
I20260811 17:14:17.654098 27693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:17.650680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59720 (local address 127.21.54.130:42125)
0811 17:14:17.651394 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:17.651407 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:17.651432 (+ 25us) server_negotiation.cc:407] Connection header received
0811 17:14:17.651488 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:17.651497 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:17.651676 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:17.652091 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:17.652945 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:17.652950 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:17.652970 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:17.652985 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:17.653241 (+ 256us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:17.653276 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:17.653799 (+ 523us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:17.653850 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 17:14:17.653918 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":172,"threads_started":1}
I20260811 17:14:17.656741 27602 tablet_service.cc:1467] Processing CreateTablet for tablet 75ffb3d16efb4d22b2a3e76d910a48c0 (DEFAULT_TABLE table=test-table [id=5e0c56694e0241c1bf5f5b039bbda5a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:17.656874 27298 tablet_service.cc:1467] Processing CreateTablet for tablet 75ffb3d16efb4d22b2a3e76d910a48c0 (DEFAULT_TABLE table=test-table [id=5e0c56694e0241c1bf5f5b039bbda5a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:17.656873 27452 tablet_service.cc:1467] Processing CreateTablet for tablet 75ffb3d16efb4d22b2a3e76d910a48c0 (DEFAULT_TABLE table=test-table [id=5e0c56694e0241c1bf5f5b039bbda5a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:17.657975 27602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75ffb3d16efb4d22b2a3e76d910a48c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:17.657991 27452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75ffb3d16efb4d22b2a3e76d910a48c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:17.657979 27298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75ffb3d16efb4d22b2a3e76d910a48c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:17.665417 27694 tablet_bootstrap.cc:492] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9: Bootstrap starting.
I20260811 17:14:17.666412 27696 tablet_bootstrap.cc:492] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f: Bootstrap starting.
I20260811 17:14:17.667014 27697 tablet_bootstrap.cc:492] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f: Bootstrap starting.
I20260811 17:14:17.668965 27694 tablet_bootstrap.cc:654] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:17.668982 27696 tablet_bootstrap.cc:654] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:17.669284 27697 tablet_bootstrap.cc:654] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:17.669862 27694 log.cc:824] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:17.669955 27696 log.cc:824] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:17.670049 27697 log.cc:824] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:17.672049 27694 tablet_bootstrap.cc:492] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9: No bootstrap required, opened a new log
I20260811 17:14:17.672099 27696 tablet_bootstrap.cc:492] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f: No bootstrap required, opened a new log
I20260811 17:14:17.672098 27697 tablet_bootstrap.cc:492] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f: No bootstrap required, opened a new log
I20260811 17:14:17.672312 27696 ts_tablet_manager.cc:1397] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260811 17:14:17.672312 27697 ts_tablet_manager.cc:1397] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 17:14:17.672317 27694 ts_tablet_manager.cc:1397] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 17:14:17.676776 27694 raft_consensus.cc:348] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1057775631e450abecbfd5516db846f" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 33877 } } peers { permanent_uuid: "58fb3a02b17446d6a6219c40b4ac1df9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38471 } } peers { permanent_uuid: "89a1211cc646483881d9f2993b141a1f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42125 } }
I20260811 17:14:17.677093 27694 raft_consensus.cc:374] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:17.677156 27694 raft_consensus.cc:729] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58fb3a02b17446d6a6219c40b4ac1df9, State: Initialized, Role: FOLLOWER
I20260811 17:14:17.677165 27697 raft_consensus.cc:348] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1057775631e450abecbfd5516db846f" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 33877 } } peers { permanent_uuid: "58fb3a02b17446d6a6219c40b4ac1df9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38471 } } peers { permanent_uuid: "89a1211cc646483881d9f2993b141a1f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42125 } }
I20260811 17:14:17.677165 27696 raft_consensus.cc:348] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1057775631e450abecbfd5516db846f" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 33877 } } peers { permanent_uuid: "58fb3a02b17446d6a6219c40b4ac1df9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38471 } } peers { permanent_uuid: "89a1211cc646483881d9f2993b141a1f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42125 } }
I20260811 17:14:17.677445 27697 raft_consensus.cc:374] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:17.677445 27696 raft_consensus.cc:374] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:17.677510 27696 raft_consensus.cc:729] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1057775631e450abecbfd5516db846f, State: Initialized, Role: FOLLOWER
I20260811 17:14:17.677506 27697 raft_consensus.cc:729] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89a1211cc646483881d9f2993b141a1f, State: Initialized, Role: FOLLOWER
I20260811 17:14:17.677707 27694 consensus_queue.cc:260] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1057775631e450abecbfd5516db846f" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 33877 } } peers { permanent_uuid: "58fb3a02b17446d6a6219c40b4ac1df9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38471 } } peers { permanent_uuid: "89a1211cc646483881d9f2993b141a1f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42125 } }
I20260811 17:14:17.677969 27696 consensus_queue.cc:260] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1057775631e450abecbfd5516db846f" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 33877 } } peers { permanent_uuid: "58fb3a02b17446d6a6219c40b4ac1df9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38471 } } peers { permanent_uuid: "89a1211cc646483881d9f2993b141a1f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42125 } }
I20260811 17:14:17.677969 27697 consensus_queue.cc:260] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1057775631e450abecbfd5516db846f" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 33877 } } peers { permanent_uuid: "58fb3a02b17446d6a6219c40b4ac1df9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38471 } } peers { permanent_uuid: "89a1211cc646483881d9f2993b141a1f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42125 } }
I20260811 17:14:17.678750 27524 heartbeater.cc:503] Master 127.21.54.190:36443 was elected leader, sending a full tablet report...
I20260811 17:14:17.679061 27375 heartbeater.cc:503] Master 127.21.54.190:36443 was elected leader, sending a full tablet report...
I20260811 17:14:17.679042 27696 ts_tablet_manager.cc:1428] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260811 17:14:17.680068 27694 ts_tablet_manager.cc:1428] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260811 17:14:17.680197 27697 ts_tablet_manager.cc:1428] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260811 17:14:17.681957 27676 heartbeater.cc:503] Master 127.21.54.190:36443 was elected leader, sending a full tablet report...
W20260811 17:14:17.710570 27377 tablet.cc:2369] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:14:17.715232 27702 raft_consensus.cc:482] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:14:17.715431 27702 raft_consensus.cc:504] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1057775631e450abecbfd5516db846f" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 33877 } } peers { permanent_uuid: "58fb3a02b17446d6a6219c40b4ac1df9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38471 } } peers { permanent_uuid: "89a1211cc646483881d9f2993b141a1f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42125 } }
I20260811 17:14:17.716653 27702 leader_election.cc:288] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58fb3a02b17446d6a6219c40b4ac1df9 (127.21.54.131:38471), 89a1211cc646483881d9f2993b141a1f (127.21.54.130:42125)
I20260811 17:14:17.718750 27378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:17.716832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:38471 (local address 127.21.54.129:51027)
0811 17:14:17.716992 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:17.717011 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:17.717126 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:17.717431 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:17.717434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:17.717470 (+ 36us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:17.718049 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:14:17.718053 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:17.718068 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:14:17.718105 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:17.718431 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:17.718435 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:17.718439 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 17:14:17.718536 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 17:14:17.718580 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 17:14:17.718842 27691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:17.717013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:51027 (local address 127.21.54.131:38471)
0811 17:14:17.717168 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:17.717176 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:17.717193 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:14:17.717243 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:17.717247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:17.717284 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:17.717368 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:17.718263 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:17.718267 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:17.718270 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:17.718284 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:17.718306 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:17.718320 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:17.718421 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:17.718640 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 17:14:17.718694 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 17:14:17.718905 27706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:17.717008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:42125 (local address 127.21.54.129:45341)
0811 17:14:17.717385 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:17.717399 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:17.717487 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:17.717801 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:17.717804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:17.717818 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:17.718239 (+ 421us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:14:17.718241 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:17.718253 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:14:17.718280 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:17.718630 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:17.718633 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:17.718637 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 17:14:17.718737 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 17:14:17.718769 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":141,"threads_started":1}
I20260811 17:14:17.718930 27693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:17.717192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:45341 (local address 127.21.54.130:42125)
0811 17:14:17.717376 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:17.717380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:17.717474 (+ 94us) server_negotiation.cc:407] Connection header received
0811 17:14:17.717610 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:17.717615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:17.717666 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:17.717784 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:17.718445 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:17.718449 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:17.718451 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:17.718466 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:17.718504 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:17.718517 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:17.718615 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:17.718752 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 17:14:17.718798 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 17:14:17.719606 27474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75ffb3d16efb4d22b2a3e76d910a48c0" candidate_uuid: "a1057775631e450abecbfd5516db846f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89a1211cc646483881d9f2993b141a1f" is_pre_election: true
I20260811 17:14:17.719668 27624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75ffb3d16efb4d22b2a3e76d910a48c0" candidate_uuid: "a1057775631e450abecbfd5516db846f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58fb3a02b17446d6a6219c40b4ac1df9" is_pre_election: true
I20260811 17:14:17.719969 27624 raft_consensus.cc:2454] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1057775631e450abecbfd5516db846f in term 0.
I20260811 17:14:17.719969 27474 raft_consensus.cc:2454] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1057775631e450abecbfd5516db846f in term 0.
I20260811 17:14:17.720564 27242 leader_election.cc:302] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89a1211cc646483881d9f2993b141a1f, a1057775631e450abecbfd5516db846f; no voters:
I20260811 17:14:17.720901 27702 raft_consensus.cc:2781] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:14:17.721014 27702 raft_consensus.cc:482] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:14:17.721092 27702 raft_consensus.cc:3037] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:17.722242 27702 raft_consensus.cc:504] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1057775631e450abecbfd5516db846f" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 33877 } } peers { permanent_uuid: "58fb3a02b17446d6a6219c40b4ac1df9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38471 } } peers { permanent_uuid: "89a1211cc646483881d9f2993b141a1f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42125 } }
I20260811 17:14:17.722645 27702 leader_election.cc:288] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f [CANDIDATE]: Term 1 election: Requested vote from peers 58fb3a02b17446d6a6219c40b4ac1df9 (127.21.54.131:38471), 89a1211cc646483881d9f2993b141a1f (127.21.54.130:42125)
I20260811 17:14:17.723167 27474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75ffb3d16efb4d22b2a3e76d910a48c0" candidate_uuid: "a1057775631e450abecbfd5516db846f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89a1211cc646483881d9f2993b141a1f"
I20260811 17:14:17.723167 27624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75ffb3d16efb4d22b2a3e76d910a48c0" candidate_uuid: "a1057775631e450abecbfd5516db846f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58fb3a02b17446d6a6219c40b4ac1df9"
I20260811 17:14:17.723320 27474 raft_consensus.cc:3037] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:17.723320 27624 raft_consensus.cc:3037] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:17.724835 27624 raft_consensus.cc:2454] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1057775631e450abecbfd5516db846f in term 1.
I20260811 17:14:17.724835 27474 raft_consensus.cc:2454] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1057775631e450abecbfd5516db846f in term 1.
I20260811 17:14:17.725227 27244 leader_election.cc:302] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58fb3a02b17446d6a6219c40b4ac1df9, a1057775631e450abecbfd5516db846f; no voters:
I20260811 17:14:17.725463 27702 raft_consensus.cc:2781] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:17.725731 27702 raft_consensus.cc:686] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f [term 1 LEADER]: Becoming Leader. State: Replica: a1057775631e450abecbfd5516db846f, State: Running, Role: LEADER
I20260811 17:14:17.726200 27702 consensus_queue.cc:237] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1057775631e450abecbfd5516db846f" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 33877 } } peers { permanent_uuid: "58fb3a02b17446d6a6219c40b4ac1df9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38471 } } peers { permanent_uuid: "89a1211cc646483881d9f2993b141a1f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42125 } }
I20260811 17:14:17.730073 27153 catalog_manager.cc:5495] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f reported cstate change: term changed from 0 to 1, leader changed from <none> to a1057775631e450abecbfd5516db846f (127.21.54.129). New cstate: current_term: 1 leader_uuid: "a1057775631e450abecbfd5516db846f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1057775631e450abecbfd5516db846f" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 33877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58fb3a02b17446d6a6219c40b4ac1df9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89a1211cc646483881d9f2993b141a1f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42125 } health_report { overall_health: UNKNOWN } } }
I20260811 17:14:17.758375 27690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:17.756452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43096 (local address 127.21.54.129:33877)
0811 17:14:17.756626 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:17.756633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:17.756653 (+ 20us) server_negotiation.cc:407] Connection header received
0811 17:14:17.756887 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:17.756890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:17.756995 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:17.757105 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:17.757921 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:17.757932 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:17.757935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:17.757950 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:17.757976 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:17.757985 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:17.758093 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:17.758172 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 17:14:17.758219 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 17:14:17.765667 27624 raft_consensus.cc:1261] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9 [term 1 FOLLOWER]: Refusing update from remote peer a1057775631e450abecbfd5516db846f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:17.765681 27474 raft_consensus.cc:1261] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f [term 1 FOLLOWER]: Refusing update from remote peer a1057775631e450abecbfd5516db846f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:17.766587 27708 consensus_queue.cc:1035] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f [LEADER]: Connected to new peer: Peer: permanent_uuid: "58fb3a02b17446d6a6219c40b4ac1df9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:17.766826 27702 consensus_queue.cc:1035] T 75ffb3d16efb4d22b2a3e76d910a48c0 P a1057775631e450abecbfd5516db846f [LEADER]: Connected to new peer: Peer: permanent_uuid: "89a1211cc646483881d9f2993b141a1f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:17.775336 27714 mvcc.cc:205] Tried to move back new op lower bound from 7317374802999136256 to 7317374802849329152. Current Snapshot: MvccSnapshot[applied={T|T < 7317374802999136256}]
I20260811 17:14:17.775791 27711 mvcc.cc:205] Tried to move back new op lower bound from 7317374802999136256 to 7317374802849329152. Current Snapshot: MvccSnapshot[applied={T|T < 7317374802999136256}]
I20260811 17:14:17.778105 27712 mvcc.cc:205] Tried to move back new op lower bound from 7317374802999136256 to 7317374802849329152. Current Snapshot: MvccSnapshot[applied={T|T < 7317374802999136256 or (T in {7317374802999136256})}]
I20260811 17:14:17.785502 27153 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:17.785737 27153 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:17.788837 27153 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fa813fb48654c5d9771528e344fe89f: Sending DeleteTablet for 3 replicas of tablet 75ffb3d16efb4d22b2a3e76d910a48c0
I20260811 17:14:17.789858 27602 tablet_service.cc:1514] Processing DeleteTablet for tablet 75ffb3d16efb4d22b2a3e76d910a48c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:17 UTC) from {username='slave'} at 127.0.0.1:47956
I20260811 17:14:17.790529 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 27225
I20260811 17:14:17.794422 27723 tablet_replica.cc:330] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9: stopping tablet replica
I20260811 17:14:17.794528 27452 tablet_service.cc:1514] Processing DeleteTablet for tablet 75ffb3d16efb4d22b2a3e76d910a48c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:17 UTC) from {username='slave'} at 127.0.0.1:59720
I20260811 17:14:17.795027 27723 raft_consensus.cc:2229] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:17.795451 27723 raft_consensus.cc:2258] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:17.795956 27725 tablet_replica.cc:330] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f: stopping tablet replica
I20260811 17:14:17.796370 27725 raft_consensus.cc:2229] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:17.796747 27725 raft_consensus.cc:2258] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:17.797629 27723 ts_tablet_manager.cc:1905] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:17.798370 27725 ts_tablet_manager.cc:1905] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:17.800081 27723 ts_tablet_manager.cc:1918] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:14:17.800215 27723 log.cc:1196] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/75ffb3d16efb4d22b2a3e76d910a48c0
I20260811 17:14:17.800577 27723 ts_tablet_manager.cc:1939] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 58fb3a02b17446d6a6219c40b4ac1df9: Deleting consensus metadata
I20260811 17:14:17.801927 27725 ts_tablet_manager.cc:1918] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:14:17.801942 27138 catalog_manager.cc:4841] TS 58fb3a02b17446d6a6219c40b4ac1df9 (127.21.54.131:38471): tablet 75ffb3d16efb4d22b2a3e76d910a48c0 (table test-table [id=5e0c56694e0241c1bf5f5b039bbda5a9]) successfully deleted
I20260811 17:14:17.802057 27725 log.cc:1196] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/75ffb3d16efb4d22b2a3e76d910a48c0
I20260811 17:14:17.802434 27725 ts_tablet_manager.cc:1939] T 75ffb3d16efb4d22b2a3e76d910a48c0 P 89a1211cc646483881d9f2993b141a1f: Deleting consensus metadata
W20260811 17:14:17.803896 27136 connection.cc:535] client connection to 127.21.54.129:33877 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 17:14:17.804068 27136 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 17:14:17.804643 27136 catalog_manager.cc:4841] TS 89a1211cc646483881d9f2993b141a1f (127.21.54.130:42125): tablet 75ffb3d16efb4d22b2a3e76d910a48c0 (table test-table [id=5e0c56694e0241c1bf5f5b039bbda5a9]) successfully deleted
I20260811 17:14:17.805058 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 27379
I20260811 17:14:17.805404 27689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:17.804976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:33877 (local address 127.0.0.1:43112)
0811 17:14:17.805203 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:17.805294 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.129:33877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 17:14:17.807789 27136 catalog_manager.cc:4584] TS a1057775631e450abecbfd5516db846f (127.21.54.129:33877): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 75ffb3d16efb4d22b2a3e76d910a48c0: Network error: Client connection negotiation failed: client connection to 127.21.54.129:33877: connect: Connection refused (error 111)
I20260811 17:14:17.815310 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 27527
I20260811 17:14:17.826224 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 27117
2026-08-11T17:14:17Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1121 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 17:14:17.852236 21722 mini_oidc.cc:152] Starting JWKS server
I20260811 17:14:17.854307 21722 webserver.cc:460] Webserver started at https://127.0.0.1:37127/ using document root <none> and password file <none>
I20260811 17:14:17.854486 21722 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 17:14:17.854749 21722 webserver.cc:460] Webserver started at http://127.0.0.1:39111/ using document root <none> and password file <none>
2026-08-11T17:14:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:14:17Z Disabled control of system clock
I20260811 17:14:17.867064 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:37647
--webserver_interface=127.21.54.190
--webserver_port=0
--builtin_ntp_servers=127.21.54.148:39247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:37647
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37127/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0/testchainca.cert with env {}
W20260811 17:14:17.988348 27732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:17.988731 27732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:17.988777 27732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:17.992990 27732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:17.993095 27732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:17.993117 27732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:17.993135 27732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:17.993151 27732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 17:14:17.993168 27732 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 17:14:17.993182 27732 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37127/jwks.json
I20260811 17:14:17.999847 27732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:37647
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:37647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37127/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:18.001144 27732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:18.002261 27732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:18.012390 27738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:18.012390 27737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:18.012460 27732 server_base.cc:1034] running on GCE node
W20260811 17:14:18.012418 27740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:18.013531 27732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:18.014834 27732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:18.016096 27732 hybrid_clock.cc:648] HybridClock initialized: now 1786468458016063 us; error 51 us; skew 500 ppm
I20260811 17:14:18.018656 27732 webserver.cc:460] Webserver started at http://127.21.54.190:32953/ using document root <none> and password file <none>
I20260811 17:14:18.019353 27732 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:18.019428 27732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:18.019876 27732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:18.022851 27732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "8f554cfda88747d8977f7887dc01a8f8"
format_stamp: "Formatted at 2026-08-11 17:14:18 on dist-test-slave-fd7l"
server_key: "4df19de08b776d7f047fb34d580d5c87"
server_key_iv: "67dc24663068f04e915a920525c48e70"
server_key_version: "encryptionkey@0"
I20260811 17:14:18.023576 27732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "8f554cfda88747d8977f7887dc01a8f8"
format_stamp: "Formatted at 2026-08-11 17:14:18 on dist-test-slave-fd7l"
server_key: "4df19de08b776d7f047fb34d580d5c87"
server_key_iv: "67dc24663068f04e915a920525c48e70"
server_key_version: "encryptionkey@0"
I20260811 17:14:18.029089 27732 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 17:14:18.032232 27747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:18.034051 27732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:18.034286 27732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "8f554cfda88747d8977f7887dc01a8f8"
format_stamp: "Formatted at 2026-08-11 17:14:18 on dist-test-slave-fd7l"
server_key: "4df19de08b776d7f047fb34d580d5c87"
server_key_iv: "67dc24663068f04e915a920525c48e70"
server_key_version: "encryptionkey@0"
I20260811 17:14:18.034436 27732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:18.069788 27732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:18.070591 27732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:18.070776 27732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:18.079546 27732 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:37647
I20260811 17:14:18.079731 27801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:37647 every 8 connection(s)
I20260811 17:14:18.080749 27732 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:14:18.084424 27802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:18.085464 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 27732
I20260811 17:14:18.085621 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:14:18.085948 21722 external_mini_cluster.cc:1428] Setting key 67dbb7caa15d47552e559967722776ad
I20260811 17:14:18.091449 27802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8: Bootstrap starting.
I20260811 17:14:18.094128 27802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:18.094920 27802 log.cc:824] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:18.096030 27807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:18.087495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59540 (local address 127.21.54.190:37647)
0811 17:14:18.088287 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:18.088304 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:18.088342 (+ 38us) server_negotiation.cc:407] Connection header received
0811 17:14:18.089090 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:18.089118 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:18.089537 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:18.089917 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:18.090389 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:18.091355 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:18.092369 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:18.092942 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:18.093144 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:18.094066 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:18.094071 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:18.094085 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:18.094578 (+ 493us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:18.094639 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:18.094661 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:18.094776 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:18.094932 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 17:14:18.095240 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":154,"threads_started":1}
I20260811 17:14:18.097402 27802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8: No bootstrap required, opened a new log
I20260811 17:14:18.099429 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:14:18.101696 27802 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f554cfda88747d8977f7887dc01a8f8" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37647 } }
I20260811 17:14:18.101977 27802 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:18.102077 27802 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f554cfda88747d8977f7887dc01a8f8, State: Initialized, Role: FOLLOWER
I20260811 17:14:18.102596 27802 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f554cfda88747d8977f7887dc01a8f8" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37647 } }
I20260811 17:14:18.102764 27802 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:14:18.102859 27802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:14:18.102974 27802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:18.104074 27802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f554cfda88747d8977f7887dc01a8f8" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37647 } }
I20260811 17:14:18.104483 27802 leader_election.cc:302] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f554cfda88747d8977f7887dc01a8f8; no voters:
I20260811 17:14:18.104925 27802 leader_election.cc:288] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:14:18.105145 27809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:18.105509 27809 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8 [term 1 LEADER]: Becoming Leader. State: Replica: 8f554cfda88747d8977f7887dc01a8f8, State: Running, Role: LEADER
I20260811 17:14:18.105940 27809 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f554cfda88747d8977f7887dc01a8f8" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37647 } }
I20260811 17:14:18.106391 27802 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:18.107921 27811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f554cfda88747d8977f7887dc01a8f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f554cfda88747d8977f7887dc01a8f8" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37647 } } }
I20260811 17:14:18.108096 27811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:18.108321 27812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f554cfda88747d8977f7887dc01a8f8. Latest consensus state: current_term: 1 leader_uuid: "8f554cfda88747d8977f7887dc01a8f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f554cfda88747d8977f7887dc01a8f8" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37647 } } }
I20260811 17:14:18.108438 27812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:18.108650 27820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:14:18.112143 27820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:14:18.118407 27820 catalog_manager.cc:1347] Generated new cluster ID: 9162340dc47243e7b6d042a3d7fc46e0
I20260811 17:14:18.118532 27820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:14:18.128482 27820 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:14:18.129767 27820 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:14:18.134593 27820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f554cfda88747d8977f7887dc01a8f8: Generated new TSK 0
I20260811 17:14:18.135390 27820 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 17:14:18.164695 27835 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 17:14:18.164870 27835 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 17:14:18.165572 27807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:18.145574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59556 (local address 127.21.54.190:37647)
0811 17:14:18.145720 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:18.145725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:18.145883 (+ 158us) server_negotiation.cc:407] Connection header received
0811 17:14:18.146118 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:18.146122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:18.146197 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:18.146321 (+ 124us) server_negotiation.cc:226] Negotiated authn=JWT
0811 17:14:18.146573 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:18.147461 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:18.148421 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:18.148810 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:18.148922 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:18.149366 (+ 444us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 17:14:18.165068 (+ 15702us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37127/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 17:14:18.165395 (+ 327us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59556: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37127/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 17:14:18.165643 27807 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59556: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37127/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 17:14:18.166956 27834 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:18.145442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:37647 (local address 127.0.0.1:59556)
0811 17:14:18.145803 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:18.145817 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:18.145937 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:18.146342 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:18.146344 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:18.146351 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0811 17:14:18.146460 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:18.146464 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:18.147601 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:18.147605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:18.148282 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:18.148290 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:18.148969 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:18.148972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:18.149077 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:18.149131 (+ 54us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 17:14:18.165341 (+ 16210us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37127/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 17:14:18.165563 (+ 222us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.54.190:37647: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37127/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":129,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.21.54.190:37647: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37127/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 17:14:18.167907 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 27732
2026-08-11T17:14:18Z chronyd exiting
I20260811 17:14:18.215368 21722 test_util.cc:175] -----------------------------------------------
I20260811 17:14:18.215517 21722 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468428054096-21722-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (365 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 17:14:18.216884 21722 mini_oidc.cc:152] Starting JWKS server
I20260811 17:14:18.217821 21722 webserver.cc:460] Webserver started at https://127.0.0.1:36909/ using document root <none> and password file <none>
I20260811 17:14:18.217943 21722 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 17:14:18.218245 21722 webserver.cc:460] Webserver started at http://127.0.0.1:42511/ using document root <none> and password file <none>
2026-08-11T17:14:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:14:18Z Disabled control of system clock
I20260811 17:14:18.230444 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:43107
--webserver_interface=127.21.54.190
--webserver_port=0
--builtin_ntp_servers=127.21.54.148:41791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:43107
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36909/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468428054096-21722-0/testchainca.cert with env {}
W20260811 17:14:18.349941 27841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:18.350253 27841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:18.350304 27841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:18.354723 27841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:18.354802 27841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:18.354820 27841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:18.354838 27841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:18.354854 27841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 17:14:18.354872 27841 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 17:14:18.354885 27841 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36909/jwks.json
I20260811 17:14:18.360674 27841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:41791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:43107
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:43107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36909/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468428054096-21722-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:18.361847 27841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:18.362921 27841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:18.372715 27849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:18.372951 27841 server_base.cc:1034] running on GCE node
W20260811 17:14:18.372773 27846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:18.372715 27847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:18.373816 27841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:18.375149 27841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:18.376375 27841 hybrid_clock.cc:648] HybridClock initialized: now 1786468458376330 us; error 60 us; skew 500 ppm
I20260811 17:14:18.378563 27841 webserver.cc:460] Webserver started at http://127.21.54.190:44543/ using document root <none> and password file <none>
I20260811 17:14:18.379223 27841 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:18.379331 27841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:18.379602 27841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:18.382202 27841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "3ede5dfbf48d47388f3c093b763f3581"
format_stamp: "Formatted at 2026-08-11 17:14:18 on dist-test-slave-fd7l"
server_key: "c8a877f8f64cf5b36a7777bce4ef2b59"
server_key_iv: "04a232e041d0a63ce1e144e600034c87"
server_key_version: "encryptionkey@0"
I20260811 17:14:18.382900 27841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "3ede5dfbf48d47388f3c093b763f3581"
format_stamp: "Formatted at 2026-08-11 17:14:18 on dist-test-slave-fd7l"
server_key: "c8a877f8f64cf5b36a7777bce4ef2b59"
server_key_iv: "04a232e041d0a63ce1e144e600034c87"
server_key_version: "encryptionkey@0"
I20260811 17:14:18.387298 27841 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 17:14:18.390199 27856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:18.391619 27841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:14:18.391784 27841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "3ede5dfbf48d47388f3c093b763f3581"
format_stamp: "Formatted at 2026-08-11 17:14:18 on dist-test-slave-fd7l"
server_key: "c8a877f8f64cf5b36a7777bce4ef2b59"
server_key_iv: "04a232e041d0a63ce1e144e600034c87"
server_key_version: "encryptionkey@0"
I20260811 17:14:18.391920 27841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:18.402976 27841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:18.403774 27841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:18.404009 27841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:18.411953 27841 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:43107
I20260811 17:14:18.412014 27917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:43107 every 8 connection(s)
I20260811 17:14:18.413143 27841 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:14:18.416709 27918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:18.418377 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 27841
I20260811 17:14:18.418506 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:14:18.418846 21722 external_mini_cluster.cc:1428] Setting key e2825dd2dc66df99405d5d96cec50173
I20260811 17:14:18.423686 27918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581: Bootstrap starting.
I20260811 17:14:18.426206 27918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:18.427175 27918 log.cc:824] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:18.429405 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:18.420411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34776 (local address 127.21.54.190:43107)
0811 17:14:18.421196 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:18.421214 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:18.421255 (+ 41us) server_negotiation.cc:407] Connection header received
0811 17:14:18.422011 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:18.422034 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:18.422501 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:18.422897 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:18.423370 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:18.424460 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:18.425467 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:18.425974 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:18.426167 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:18.427133 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:18.427139 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:18.427154 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:18.427686 (+ 532us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:18.427768 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:18.427801 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:18.427966 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:18.428144 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 17:14:18.428488 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":151,"threads_started":1}
I20260811 17:14:18.429693 27918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581: No bootstrap required, opened a new log
I20260811 17:14:18.432751 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:14:18.434275 27918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ede5dfbf48d47388f3c093b763f3581" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43107 } }
I20260811 17:14:18.434572 27918 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:18.434638 27918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ede5dfbf48d47388f3c093b763f3581, State: Initialized, Role: FOLLOWER
I20260811 17:14:18.435165 27918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ede5dfbf48d47388f3c093b763f3581" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43107 } }
I20260811 17:14:18.435307 27918 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:14:18.435380 27918 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:14:18.435487 27918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:18.436651 27918 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ede5dfbf48d47388f3c093b763f3581" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43107 } }
I20260811 17:14:18.437073 27918 leader_election.cc:302] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ede5dfbf48d47388f3c093b763f3581; no voters:
I20260811 17:14:18.437425 27918 leader_election.cc:288] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:14:18.437697 27923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:18.437999 27923 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581 [term 1 LEADER]: Becoming Leader. State: Replica: 3ede5dfbf48d47388f3c093b763f3581, State: Running, Role: LEADER
I20260811 17:14:18.438338 27923 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ede5dfbf48d47388f3c093b763f3581" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43107 } }
I20260811 17:14:18.439075 27918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:18.440289 27925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ede5dfbf48d47388f3c093b763f3581. Latest consensus state: current_term: 1 leader_uuid: "3ede5dfbf48d47388f3c093b763f3581" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ede5dfbf48d47388f3c093b763f3581" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43107 } } }
I20260811 17:14:18.440428 27925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:18.440783 27924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ede5dfbf48d47388f3c093b763f3581" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ede5dfbf48d47388f3c093b763f3581" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43107 } } }
I20260811 17:14:18.440888 27924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:18.440984 27933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:14:18.444375 27933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:14:18.451874 27933 catalog_manager.cc:1347] Generated new cluster ID: 52c61e278cc24e86bdbedc89f64c919d
I20260811 17:14:18.451970 27933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:14:18.475112 27933 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:14:18.476555 27933 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:14:18.484455 27933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ede5dfbf48d47388f3c093b763f3581: Generated new TSK 0
I20260811 17:14:18.485479 27933 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 17:14:18.515133 27950 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 17:14:18.515261 27950 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 17:14:18.516024 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:18.499355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34784 (local address 127.21.54.190:43107)
0811 17:14:18.499512 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:18.499517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:18.499662 (+ 145us) server_negotiation.cc:407] Connection header received
0811 17:14:18.499837 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:18.499841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:18.499904 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:18.500050 (+ 146us) server_negotiation.cc:226] Negotiated authn=JWT
0811 17:14:18.500301 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:18.501200 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:18.502114 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:18.502491 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:18.502606 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:18.502953 (+ 347us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 17:14:18.515543 (+ 12590us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36909/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 17:14:18.515864 (+ 321us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34784: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36909/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 17:14:18.516112 27921 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34784: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36909/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 17:14:18.516176 27949 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:18.499249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43107 (local address 127.0.0.1:34784)
0811 17:14:18.499551 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:18.499565 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:18.499689 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:18.500043 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:18.500045 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:18.500051 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0811 17:14:18.500184 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:18.500189 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:18.501338 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:18.501341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:18.501985 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:18.501993 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:18.502645 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:18.502648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:18.502716 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:18.502744 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 17:14:18.515788 (+ 13044us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36909/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 17:14:18.515966 (+ 178us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.54.190:43107: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36909/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":113,"threads_started":1}
I20260811 17:14:18.517007 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 27841
2026-08-11T17:14:18Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (346 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 17:14:18.562896 21722 mini_oidc.cc:152] Starting JWKS server
I20260811 17:14:18.563860 21722 webserver.cc:460] Webserver started at https://127.0.0.1:33643/ using document root <none> and password file <none>
I20260811 17:14:18.563953 21722 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 17:14:18.564174 21722 webserver.cc:460] Webserver started at http://127.0.0.1:39963/ using document root <none> and password file <none>
2026-08-11T17:14:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:14:18Z Disabled control of system clock
I20260811 17:14:18.575659 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:39659
--webserver_interface=127.21.54.190
--webserver_port=0
--builtin_ntp_servers=127.21.54.148:40595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:39659
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33643/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 17:14:18.693219 27956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:18.693511 27956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:18.693552 27956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:18.697626 27956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:18.697695 27956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:18.697710 27956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:18.697727 27956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:18.697742 27956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 17:14:18.697759 27956 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 17:14:18.697772 27956 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33643/jwks.json
I20260811 17:14:18.702731 27956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:40595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:39659
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:39659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33643/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:18.703917 27956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:18.705482 27956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:18.714766 27963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:18.714772 27965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:18.714784 27969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:18.714854 27956 server_base.cc:1034] running on GCE node
I20260811 17:14:18.715804 27956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:18.717149 27956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:18.718325 27956 hybrid_clock.cc:648] HybridClock initialized: now 1786468458718305 us; error 38 us; skew 500 ppm
I20260811 17:14:18.720610 27956 webserver.cc:460] Webserver started at http://127.21.54.190:36651/ using document root <none> and password file <none>
I20260811 17:14:18.721249 27956 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:18.721319 27956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:18.721553 27956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:18.723793 27956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "300bf1b6dfc147eab1f3477e6f83e03f"
format_stamp: "Formatted at 2026-08-11 17:14:18 on dist-test-slave-fd7l"
server_key: "8826d4c53a338bfec01dac7663790782"
server_key_iv: "1dd28bef5b27d74bae3fae163e66bbbd"
server_key_version: "encryptionkey@0"
I20260811 17:14:18.724655 27956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "300bf1b6dfc147eab1f3477e6f83e03f"
format_stamp: "Formatted at 2026-08-11 17:14:18 on dist-test-slave-fd7l"
server_key: "8826d4c53a338bfec01dac7663790782"
server_key_iv: "1dd28bef5b27d74bae3fae163e66bbbd"
server_key_version: "encryptionkey@0"
I20260811 17:14:18.728839 27956 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:14:18.731570 27975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:18.733121 27956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:18.733285 27956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "300bf1b6dfc147eab1f3477e6f83e03f"
format_stamp: "Formatted at 2026-08-11 17:14:18 on dist-test-slave-fd7l"
server_key: "8826d4c53a338bfec01dac7663790782"
server_key_iv: "1dd28bef5b27d74bae3fae163e66bbbd"
server_key_version: "encryptionkey@0"
I20260811 17:14:18.733424 27956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:18.751614 27956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:18.752560 27956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:18.752785 27956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:18.760644 27956 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:39659
I20260811 17:14:18.760718 28036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:39659 every 8 connection(s)
I20260811 17:14:18.761754 27956 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:14:18.762862 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 27956
I20260811 17:14:18.763043 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:14:18.763350 21722 external_mini_cluster.cc:1428] Setting key a20cfeef1019a1d4ea37865c49532da8
I20260811 17:14:18.765828 28037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:18.772893 28037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f: Bootstrap starting.
I20260811 17:14:18.773819 28040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:18.764870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55748 (local address 127.21.54.190:39659)
0811 17:14:18.765580 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:18.765597 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:18.765635 (+ 38us) server_negotiation.cc:407] Connection header received
0811 17:14:18.766378 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:18.766398 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:18.766835 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:18.767139 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:18.767653 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:18.768749 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:18.770019 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:18.770497 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:18.770674 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:18.771683 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:18.771688 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:18.771701 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:18.772151 (+ 450us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:18.772213 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:18.772234 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:18.772320 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:18.772669 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 17:14:18.772969 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":169,"threads_started":1}
I20260811 17:14:18.775544 28037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:18.776474 28037 log.cc:824] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:18.777176 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:14:18.779109 28037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f: No bootstrap required, opened a new log
I20260811 17:14:18.783180 28037 raft_consensus.cc:348] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300bf1b6dfc147eab1f3477e6f83e03f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39659 } }
I20260811 17:14:18.783437 28037 raft_consensus.cc:374] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:18.783473 28037 raft_consensus.cc:729] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 300bf1b6dfc147eab1f3477e6f83e03f, State: Initialized, Role: FOLLOWER
I20260811 17:14:18.783914 28037 consensus_queue.cc:260] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300bf1b6dfc147eab1f3477e6f83e03f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39659 } }
I20260811 17:14:18.784045 28037 raft_consensus.cc:388] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:14:18.784121 28037 raft_consensus.cc:482] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:14:18.784204 28037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:18.785316 28037 raft_consensus.cc:504] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300bf1b6dfc147eab1f3477e6f83e03f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39659 } }
I20260811 17:14:18.785645 28037 leader_election.cc:302] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 300bf1b6dfc147eab1f3477e6f83e03f; no voters:
I20260811 17:14:18.786023 28037 leader_election.cc:288] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:14:18.786290 28042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:18.786836 28042 raft_consensus.cc:686] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f [term 1 LEADER]: Becoming Leader. State: Replica: 300bf1b6dfc147eab1f3477e6f83e03f, State: Running, Role: LEADER
I20260811 17:14:18.787194 28042 consensus_queue.cc:237] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300bf1b6dfc147eab1f3477e6f83e03f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39659 } }
I20260811 17:14:18.787568 28037 sys_catalog.cc:564] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:18.789199 28043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "300bf1b6dfc147eab1f3477e6f83e03f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300bf1b6dfc147eab1f3477e6f83e03f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39659 } } }
I20260811 17:14:18.789229 28044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 300bf1b6dfc147eab1f3477e6f83e03f. Latest consensus state: current_term: 1 leader_uuid: "300bf1b6dfc147eab1f3477e6f83e03f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300bf1b6dfc147eab1f3477e6f83e03f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39659 } } }
I20260811 17:14:18.789341 28043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:18.789354 28044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:18.789819 28054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:14:18.793123 28054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:14:18.800203 28054 catalog_manager.cc:1347] Generated new cluster ID: 808f3c006a0544c699c7c8c7ee0ce914
I20260811 17:14:18.800300 28054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:14:18.808919 28054 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:14:18.810423 28054 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:14:18.820940 28054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 300bf1b6dfc147eab1f3477e6f83e03f: Generated new TSK 0
I20260811 17:14:18.822041 28054 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:14:18.846700 28040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:18.840875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55756 (local address 127.21.54.190:39659)
0811 17:14:18.841033 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:18.841039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:18.841183 (+ 144us) server_negotiation.cc:407] Connection header received
0811 17:14:18.841359 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:18.841363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:18.841433 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:18.841561 (+ 128us) server_negotiation.cc:226] Negotiated authn=JWT
0811 17:14:18.841813 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:18.842732 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:18.843702 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:18.844071 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:18.844187 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:18.844562 (+ 375us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 17:14:18.846255 (+ 1693us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33643/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 17:14:18.846529 (+ 274us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55756: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33643/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 17:14:18.846793 28040 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55756: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33643/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 17:14:18.848204 28068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:18.840774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:39659 (local address 127.0.0.1:55756)
0811 17:14:18.841072 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:18.841091 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:18.841194 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:18.841574 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:18.841577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:18.841590 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0811 17:14:18.841685 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:18.841690 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:18.842903 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:18.842906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:18.843555 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:18.843565 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:18.844217 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:18.844220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:18.844289 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:18.844316 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 17:14:18.846442 (+ 2126us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33643/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 17:14:18.848013 (+ 1571us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.54.190:39659: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33643/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":1355,"thread_start_us":118,"threads_started":1}
I20260811 17:14:18.848937 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 27956
E20260811 17:14:18.861521 28069 webserver.cc:533] Webserver: sslize failed:
E20260811 17:14:18.861652 28069 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T17:14:18Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (330 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T17:14:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:14:18Z Disabled control of system clock
I20260811 17:14:18.905282 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:45017
--webserver_interface=127.21.54.190
--webserver_port=0
--builtin_ntp_servers=127.21.54.148:43075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:45017
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468428054096-21722-0/testchainca.cert with env {}
W20260811 17:14:19.033982 28073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:19.034305 28073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:19.034350 28073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:19.038839 28073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:19.038928 28073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:19.038944 28073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:19.038962 28073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:19.038977 28073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 17:14:19.038995 28073 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 17:14:19.039043 28073 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 17:14:19.045274 28073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:43075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:45017
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:45017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468428054096-21722-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:19.046557 28073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:19.047695 28073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:19.057407 28080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:19.057518 28083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:19.057541 28073 server_base.cc:1034] running on GCE node
W20260811 17:14:19.057776 28079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:19.058457 28073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:19.059636 28073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:19.060820 28073 hybrid_clock.cc:648] HybridClock initialized: now 1786468459060806 us; error 30 us; skew 500 ppm
I20260811 17:14:19.062987 28073 webserver.cc:460] Webserver started at http://127.21.54.190:35705/ using document root <none> and password file <none>
I20260811 17:14:19.063613 28073 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:19.063683 28073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:19.063911 28073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:19.066198 28073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "456d4c7a9d94429f8b3a59978126988d"
format_stamp: "Formatted at 2026-08-11 17:14:19 on dist-test-slave-fd7l"
server_key: "7219b5b829dc393468690369a2be25ed"
server_key_iv: "8c99b4ea0bcee2ca1a53c49d9de4361a"
server_key_version: "encryptionkey@0"
I20260811 17:14:19.066817 28073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "456d4c7a9d94429f8b3a59978126988d"
format_stamp: "Formatted at 2026-08-11 17:14:19 on dist-test-slave-fd7l"
server_key: "7219b5b829dc393468690369a2be25ed"
server_key_iv: "8c99b4ea0bcee2ca1a53c49d9de4361a"
server_key_version: "encryptionkey@0"
I20260811 17:14:19.070811 28073 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:14:19.073459 28090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:19.074869 28073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:19.075066 28073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "456d4c7a9d94429f8b3a59978126988d"
format_stamp: "Formatted at 2026-08-11 17:14:19 on dist-test-slave-fd7l"
server_key: "7219b5b829dc393468690369a2be25ed"
server_key_iv: "8c99b4ea0bcee2ca1a53c49d9de4361a"
server_key_version: "encryptionkey@0"
I20260811 17:14:19.075204 28073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:19.088716 28073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:19.089564 28073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:19.089819 28073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:19.097574 28073 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:45017
I20260811 17:14:19.097626 28150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:45017 every 8 connection(s)
I20260811 17:14:19.098950 28073 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:14:19.102523 28151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:19.103150 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 28073
I20260811 17:14:19.103291 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:14:19.103653 21722 external_mini_cluster.cc:1428] Setting key 58339f9203f6131e4243294388940fc7
I20260811 17:14:19.109736 28151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d: Bootstrap starting.
I20260811 17:14:19.112501 28151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:19.113560 28151 log.cc:824] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:19.113911 28154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:19.105299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56608 (local address 127.21.54.190:45017)
0811 17:14:19.106062 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:19.106079 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:19.106117 (+ 38us) server_negotiation.cc:407] Connection header received
0811 17:14:19.106835 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:19.106855 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:19.107307 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:19.107681 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:19.108147 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:19.109235 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:19.110256 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:19.110754 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:19.110957 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:19.111917 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:19.111922 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:19.111936 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:19.112382 (+ 446us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:19.112443 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:19.112464 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:19.112627 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:19.112800 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 17:14:19.113084 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":160,"threads_started":1}
I20260811 17:14:19.116127 28151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d: No bootstrap required, opened a new log
I20260811 17:14:19.117303 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:14:19.120599 28151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456d4c7a9d94429f8b3a59978126988d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45017 } }
I20260811 17:14:19.120894 28151 raft_consensus.cc:374] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:19.120968 28151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456d4c7a9d94429f8b3a59978126988d, State: Initialized, Role: FOLLOWER
I20260811 17:14:19.121399 28151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456d4c7a9d94429f8b3a59978126988d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45017 } }
I20260811 17:14:19.121565 28151 raft_consensus.cc:388] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:14:19.121655 28151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:14:19.121759 28151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:19.122875 28151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456d4c7a9d94429f8b3a59978126988d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45017 } }
I20260811 17:14:19.123241 28151 leader_election.cc:302] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 456d4c7a9d94429f8b3a59978126988d; no voters:
I20260811 17:14:19.123656 28151 leader_election.cc:288] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:14:19.123816 28156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:19.124094 28156 raft_consensus.cc:686] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d [term 1 LEADER]: Becoming Leader. State: Replica: 456d4c7a9d94429f8b3a59978126988d, State: Running, Role: LEADER
I20260811 17:14:19.124429 28156 consensus_queue.cc:237] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456d4c7a9d94429f8b3a59978126988d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45017 } }
I20260811 17:14:19.125087 28151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:19.126361 28157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "456d4c7a9d94429f8b3a59978126988d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456d4c7a9d94429f8b3a59978126988d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45017 } } }
I20260811 17:14:19.126331 28158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 456d4c7a9d94429f8b3a59978126988d. Latest consensus state: current_term: 1 leader_uuid: "456d4c7a9d94429f8b3a59978126988d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456d4c7a9d94429f8b3a59978126988d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45017 } } }
I20260811 17:14:19.126480 28157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:19.126488 28158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:19.126986 28166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:14:19.130656 28166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:14:19.137086 28166 catalog_manager.cc:1347] Generated new cluster ID: e91c7f2c888e4d87a821531791bfd127
I20260811 17:14:19.137181 28166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:14:19.149700 28166 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:14:19.150669 28166 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:14:19.162207 28166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 456d4c7a9d94429f8b3a59978126988d: Generated new TSK 0
I20260811 17:14:19.163204 28166 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:14:19.207333 28154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:19.178519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56618 (local address 127.21.54.190:45017)
0811 17:14:19.178682 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:19.178688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:19.178905 (+ 217us) server_negotiation.cc:407] Connection header received
0811 17:14:19.179086 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:19.179089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:19.179166 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:19.179298 (+ 132us) server_negotiation.cc:226] Negotiated authn=JWT
0811 17:14:19.179782 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:19.180686 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:19.181578 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:19.181935 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:19.182051 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:19.182399 (+ 348us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 17:14:19.206804 (+ 24405us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 17:14:19.207174 (+ 370us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56618: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 17:14:19.207434 28154 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56618: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 17:14:19.207543 28184 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:19.178479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45017 (local address 127.0.0.1:56618)
0811 17:14:19.178810 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:19.178824 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:19.178930 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:19.179397 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:19.179400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:19.179411 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0811 17:14:19.179667 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:19.179675 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:19.180814 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:19.180817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:19.181450 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:19.181458 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:19.182077 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:19.182081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:19.182173 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:19.182200 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 17:14:19.207106 (+ 24906us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 17:14:19.207319 (+ 213us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.54.190:45017: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":147,"threads_started":1}
I20260811 17:14:19.208284 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 28073
2026-08-11T17:14:19Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (337 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (136 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (140 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:14:19 dist-test-slave-fd7l krb5kdc[28198](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:14:19 dist-test-slave-fd7l krb5kdc[28198](info): set up 2 sockets
Aug 11 17:14:19 dist-test-slave-fd7l krb5kdc[28198](info): commencing operation
krb5kdc: starting...
W20260811 17:14:20.682191 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:14:20 dist-test-slave-fd7l krb5kdc[28198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:14:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:14:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:14:20.832038 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:39855
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:41879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:39855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:20.952574 28211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:20.952970 28211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:20.953061 28211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:20.957365 28211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:20.957499 28211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:20.957538 28211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:20.957571 28211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:20.957602 28211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:14:20.963901 28211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:41879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:39855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:39855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:20.965294 28211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:20.966503 28211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:20.976033 28220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:20.976173 28218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:20.976037 28216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:20.976745 28211 server_base.cc:1034] running on GCE node
I20260811 17:14:20.977411 28211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:20.978781 28211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:20.979969 28211 hybrid_clock.cc:648] HybridClock initialized: now 1786468460979949 us; error 38 us; skew 500 ppm
Aug 11 17:14:20 dist-test-slave-fd7l krb5kdc[28198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468460, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:20.987797 28211 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:14:20.989113 28211 webserver.cc:460] Webserver started at http://127.21.54.190:37469/ using document root <none> and password file <none>
I20260811 17:14:20.989809 28211 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:20.989880 28211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:20.990144 28211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:20.992584 28211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "9ba52cc217214a6bbb812bcb6a456a77"
format_stamp: "Formatted at 2026-08-11 17:14:20 on dist-test-slave-fd7l"
server_key: "322440171b2c8a40bab27be5402363d4"
server_key_iv: "ccf12f10fc8dcac6bc716064d8e12ff3"
server_key_version: "encryptionkey@0"
I20260811 17:14:20.993263 28211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "9ba52cc217214a6bbb812bcb6a456a77"
format_stamp: "Formatted at 2026-08-11 17:14:20 on dist-test-slave-fd7l"
server_key: "322440171b2c8a40bab27be5402363d4"
server_key_iv: "ccf12f10fc8dcac6bc716064d8e12ff3"
server_key_version: "encryptionkey@0"
I20260811 17:14:20.997803 28211 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:14:21.000797 28228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:21.002250 28211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 17:14:21.002413 28211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "9ba52cc217214a6bbb812bcb6a456a77"
format_stamp: "Formatted at 2026-08-11 17:14:20 on dist-test-slave-fd7l"
server_key: "322440171b2c8a40bab27be5402363d4"
server_key_iv: "ccf12f10fc8dcac6bc716064d8e12ff3"
server_key_version: "encryptionkey@0"
I20260811 17:14:21.002542 28211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:21.019647 28211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:21.023697 28211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:21.024052 28211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:21.032249 28211 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:39855
I20260811 17:14:21.032272 28288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:39855 every 8 connection(s)
I20260811 17:14:21.033412 28211 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:14:21.036470 28289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:21.040979 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 28211
I20260811 17:14:21.041113 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:14:21.041427 21722 external_mini_cluster.cc:1428] Setting key 180e6a3d3106a06a909851cf6a0949fe
I20260811 17:14:21.043337 28289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77: Bootstrap starting.
I20260811 17:14:21.046005 28289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:21.046906 28289 log.cc:824] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:21.049429 28289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77: No bootstrap required, opened a new log
Aug 11 17:14:21 dist-test-slave-fd7l krb5kdc[28198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:21.054629 28289 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba52cc217214a6bbb812bcb6a456a77" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39855 } }
I20260811 17:14:21.054929 28289 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:21.054970 28289 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ba52cc217214a6bbb812bcb6a456a77, State: Initialized, Role: FOLLOWER
I20260811 17:14:21.055580 28289 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba52cc217214a6bbb812bcb6a456a77" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39855 } }
I20260811 17:14:21.055768 28289 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:14:21.055872 28289 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:14:21.056035 28289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:21.057349 28289 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba52cc217214a6bbb812bcb6a456a77" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39855 } }
I20260811 17:14:21.057806 28289 leader_election.cc:302] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ba52cc217214a6bbb812bcb6a456a77; no voters:
I20260811 17:14:21.058287 28289 leader_election.cc:288] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:14:21.058429 28294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:21.058686 28294 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77 [term 1 LEADER]: Becoming Leader. State: Replica: 9ba52cc217214a6bbb812bcb6a456a77, State: Running, Role: LEADER
I20260811 17:14:21.059108 28294 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba52cc217214a6bbb812bcb6a456a77" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39855 } }
I20260811 17:14:21.059798 28289 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:21.059823 28292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:21.043214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35698 (local address 127.21.54.190:39855)
0811 17:14:21.043798 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:21.043816 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:21.043858 (+ 42us) server_negotiation.cc:407] Connection header received
0811 17:14:21.044743 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:21.044768 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:21.045152 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:21.045557 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:21.046582 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.047770 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.048840 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.049378 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.049585 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:21.052882 (+ 3297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:21.052888 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:21.052904 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:21.052941 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:21.055360 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:21.056018 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:21.056026 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:21.056033 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:21.056214 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:21.056631 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:21.056634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:21.056636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:21.057128 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:21.057310 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:21.057833 (+ 523us) server_negotiation.cc:299] Negotiation successful
0811 17:14:21.058237 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":213,"threads_started":1}
I20260811 17:14:21.061091 28296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ba52cc217214a6bbb812bcb6a456a77. Latest consensus state: current_term: 1 leader_uuid: "9ba52cc217214a6bbb812bcb6a456a77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba52cc217214a6bbb812bcb6a456a77" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39855 } } }
I20260811 17:14:21.061249 28296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:21.061601 28295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ba52cc217214a6bbb812bcb6a456a77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba52cc217214a6bbb812bcb6a456a77" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39855 } } }
I20260811 17:14:21.061702 28295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77 [sys.catalog]: This master's current role is: LEADER
W20260811 17:14:21.065717 28312 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:14:21.065809 28312 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:14:21.065907 28305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:14:21.066794 28305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:14:21.072794 28305 catalog_manager.cc:1347] Generated new cluster ID: daeeeb5ae825476ba5712102ac749ec3
I20260811 17:14:21.072877 28305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:14:21.084940 28305 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:14:21.086068 28305 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:14:21.092957 28305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ba52cc217214a6bbb812bcb6a456a77: Generated new TSK 0
I20260811 17:14:21.093787 28305 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
I20260811 17:14:21.129102 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:39855
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:41879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:21.258862 28319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:21.259225 28319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:21.259303 28319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:21.264047 28319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:21.264161 28319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:21.264264 28319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:14:21.269977 28319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:41879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:39855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:21.271202 28319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:21.272298 28319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:21.282918 28325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:21.282908 28326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:21.282908 28328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:21.283303 28319 server_base.cc:1034] running on GCE node
I20260811 17:14:21.283727 28319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:21.284487 28319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:21.285746 28319 hybrid_clock.cc:648] HybridClock initialized: now 1786468461285705 us; error 55 us; skew 500 ppm
Aug 11 17:14:21 dist-test-slave-fd7l krb5kdc[28198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468461, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:21.289166 28319 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:14:21.290264 28319 webserver.cc:460] Webserver started at http://127.21.54.129:32769/ using document root <none> and password file <none>
I20260811 17:14:21.290904 28319 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:21.291014 28319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:21.291272 28319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:21.293551 28319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "23a8459719b94623ad9addce2aec8a1d"
format_stamp: "Formatted at 2026-08-11 17:14:21 on dist-test-slave-fd7l"
server_key: "ee2ddf0f9a27a57f31a13a962d14f83f"
server_key_iv: "1554e7d794a2de9d6d3820f643903bdb"
server_key_version: "encryptionkey@0"
I20260811 17:14:21.294186 28319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "23a8459719b94623ad9addce2aec8a1d"
format_stamp: "Formatted at 2026-08-11 17:14:21 on dist-test-slave-fd7l"
server_key: "ee2ddf0f9a27a57f31a13a962d14f83f"
server_key_iv: "1554e7d794a2de9d6d3820f643903bdb"
server_key_version: "encryptionkey@0"
I20260811 17:14:21.298338 28319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:14:21.301542 28337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:21.302943 28319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 17:14:21.303122 28319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "23a8459719b94623ad9addce2aec8a1d"
format_stamp: "Formatted at 2026-08-11 17:14:21 on dist-test-slave-fd7l"
server_key: "ee2ddf0f9a27a57f31a13a962d14f83f"
server_key_iv: "1554e7d794a2de9d6d3820f643903bdb"
server_key_version: "encryptionkey@0"
I20260811 17:14:21.303265 28319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:21.318212 28319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:21.321985 28319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:21.322240 28319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:21.322912 28319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:21.323844 28319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:21.323935 28319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:21.324031 28319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:21.324084 28319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:21.334146 28319 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:44593
I20260811 17:14:21.334170 28471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:44593 every 8 connection(s)
I20260811 17:14:21.335132 28319 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:14:21.339685 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 28319
I20260811 17:14:21.339843 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:14:21.340154 21722 external_mini_cluster.cc:1428] Setting key c407f525b00d8f551b8b10bc073ed215
Aug 11 17:14:21 dist-test-slave-fd7l krb5kdc[28198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468461, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.54.130@KRBTEST.COM; defaulting to no policy
I20260811 17:14:21.351941 28292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:21.337404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:36753 (local address 127.21.54.190:39855)
0811 17:14:21.337536 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:21.337541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:21.338512 (+ 971us) server_negotiation.cc:407] Connection header received
0811 17:14:21.340027 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:21.340031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:21.340130 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:21.340247 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:21.341689 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.342462 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.343797 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.344122 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.344212 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:21.347606 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:21.347610 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:21.347613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:21.347642 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:21.349421 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:21.350134 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:21.350142 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:21.350144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:21.350230 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:21.350660 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:21.350663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:21.350665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:21.350870 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:21.350986 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:21.351620 (+ 634us) server_negotiation.cc:299] Negotiation successful
0811 17:14:21.351766 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 17:14:21.353005 28474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:21.337810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:39855 (local address 127.21.54.129:36753)
0811 17:14:21.338326 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:21.338372 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:21.339772 (+ 1400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:21.340437 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:21.340445 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:21.340976 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:21.341489 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:21.341539 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.342638 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.342641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:21.343612 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:21.343622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.344308 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.344314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:21.344601 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:21.345364 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:21.345403 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:21.347388 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:21.349626 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:21.349633 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:21.349658 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:21.349951 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:21.350346 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:21.350352 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:21.350354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:21.350538 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:21.350986 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:21.350996 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:21.351350 (+ 354us) client_negotiation.cc:769] Sending connection context
0811 17:14:21.351573 (+ 223us) client_negotiation.cc:240] Negotiation successful
0811 17:14:21.351950 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":119,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.130@KRBTEST.COM" created.
I20260811 17:14:21.354674 28472 heartbeater.cc:347] Connected to a master server at 127.21.54.190:39855
I20260811 17:14:21.354979 28472 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:21.355599 28472 heartbeater.cc:511] Master 127.21.54.190:39855 requested a full tablet report, sending...
I20260811 17:14:21.357508 28248 ts_manager.cc:194] Registered new tserver with Master: 23a8459719b94623ad9addce2aec8a1d (127.21.54.129:44593)
I20260811 17:14:21.358891 28248 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:36753
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
I20260811 17:14:21.374082 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:39855
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:41879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:21.498801 28477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:21.499150 28477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:21.499228 28477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:21.503427 28477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:21.503537 28477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:21.503636 28477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:14:21.509562 28477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:41879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:39855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:21.510890 28477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:21.512044 28477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:21.522648 28488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:21.522635 28484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:21.522635 28486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:21.523187 28477 server_base.cc:1034] running on GCE node
I20260811 17:14:21.523665 28477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:21.524416 28477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:21.525648 28477 hybrid_clock.cc:648] HybridClock initialized: now 1786468461525603 us; error 59 us; skew 500 ppm
Aug 11 17:14:21 dist-test-slave-fd7l krb5kdc[28198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468461, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:21.529203 28477 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:14:21.530330 28477 webserver.cc:460] Webserver started at http://127.21.54.130:43915/ using document root <none> and password file <none>
I20260811 17:14:21.530982 28477 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:21.531050 28477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:21.531314 28477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:21.533727 28477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "0245116090ec4fe994f86d310359714a"
format_stamp: "Formatted at 2026-08-11 17:14:21 on dist-test-slave-fd7l"
server_key: "a57f66b78444614bcf3aeb7edce24890"
server_key_iv: "7b5ab157e37f40002511e88e61c17a80"
server_key_version: "encryptionkey@0"
I20260811 17:14:21.534348 28477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "0245116090ec4fe994f86d310359714a"
format_stamp: "Formatted at 2026-08-11 17:14:21 on dist-test-slave-fd7l"
server_key: "a57f66b78444614bcf3aeb7edce24890"
server_key_iv: "7b5ab157e37f40002511e88e61c17a80"
server_key_version: "encryptionkey@0"
I20260811 17:14:21.538698 28477 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 17:14:21.541774 28495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:21.543282 28477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:14:21.543399 28477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "0245116090ec4fe994f86d310359714a"
format_stamp: "Formatted at 2026-08-11 17:14:21 on dist-test-slave-fd7l"
server_key: "a57f66b78444614bcf3aeb7edce24890"
server_key_iv: "7b5ab157e37f40002511e88e61c17a80"
server_key_version: "encryptionkey@0"
I20260811 17:14:21.543545 28477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:21.571594 28477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:21.575321 28477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:21.575495 28477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:21.576093 28477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:21.577081 28477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:21.577131 28477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:21.577175 28477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:21.577190 28477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:21.586969 28477 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:40733
I20260811 17:14:21.586997 28626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:40733 every 8 connection(s)
I20260811 17:14:21.588043 28477 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:14:21.593189 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 28477
I20260811 17:14:21.593343 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:14:21.593658 21722 external_mini_cluster.cc:1428] Setting key 8f554c9dae6e4b61e510c154f6c862ba
Aug 11 17:14:21 dist-test-slave-fd7l krb5kdc[28198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468461, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:21.604493 28292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:21.590215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:37309 (local address 127.21.54.190:39855)
0811 17:14:21.590396 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:21.590399 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:21.591367 (+ 968us) server_negotiation.cc:407] Connection header received
0811 17:14:21.593058 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:21.593083 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:21.593148 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:21.593267 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:21.594917 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.595451 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.596697 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.596929 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.597046 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:21.600126 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:21.600134 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:21.600140 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:21.600178 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:21.602082 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:21.602679 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:21.602683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:21.602685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:21.602784 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:21.603170 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:21.603174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:21.603176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:21.603445 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:21.603570 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:21.604124 (+ 554us) server_negotiation.cc:299] Negotiation successful
0811 17:14:21.604266 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
WARNING: no policy specified for kudu/127.21.54.131@KRBTEST.COM; defaulting to no policy
I20260811 17:14:21.605404 28629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:21.590649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:39855 (local address 127.21.54.130:37309)
0811 17:14:21.591193 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:21.591239 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:21.592747 (+ 1508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:21.593487 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:21.593496 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:21.594019 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:21.594723 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:21.594741 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.595594 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.595597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:21.596498 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:21.596577 (+ 79us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.597089 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.597093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:21.597341 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:21.598031 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:21.598057 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:21.599895 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:21.602240 (+ 2345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:21.602246 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:21.602262 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:21.602564 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:21.602884 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:21.602886 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:21.602887 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:21.603062 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:21.603575 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:21.603581 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:21.603849 (+ 268us) client_negotiation.cc:769] Sending connection context
0811 17:14:21.604044 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 17:14:21.604330 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":134,"threads_started":1}
I20260811 17:14:21.606748 28627 heartbeater.cc:347] Connected to a master server at 127.21.54.190:39855
I20260811 17:14:21.607079 28627 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:21.607688 28627 heartbeater.cc:511] Master 127.21.54.190:39855 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.131@KRBTEST.COM" created.
I20260811 17:14:21.609071 28248 ts_manager.cc:194] Registered new tserver with Master: 0245116090ec4fe994f86d310359714a (127.21.54.130:40733)
I20260811 17:14:21.609956 28248 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:37309
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
I20260811 17:14:21.628501 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:39855
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:41879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:21.753470 28632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:21.753796 28632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:21.753875 28632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:21.758050 28632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:21.758139 28632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:21.758229 28632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:14:21.764020 28632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:41879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:39855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:21.765269 28632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:21.766482 28632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:21.776957 28641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:21.777159 28632 server_base.cc:1034] running on GCE node
W20260811 17:14:21.776959 28638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:21.777333 28639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:21.777822 28632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:21.778527 28632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:21.779745 28632 hybrid_clock.cc:648] HybridClock initialized: now 1786468461779728 us; error 29 us; skew 500 ppm
Aug 11 17:14:21 dist-test-slave-fd7l krb5kdc[28198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468461, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:21.783144 28632 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:14:21.784147 28632 webserver.cc:460] Webserver started at http://127.21.54.131:35289/ using document root <none> and password file <none>
I20260811 17:14:21.784919 28632 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:21.785019 28632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:21.785266 28632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:21.787539 28632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "16e20e18f81b4dee8c466630c5e2bfbe"
format_stamp: "Formatted at 2026-08-11 17:14:21 on dist-test-slave-fd7l"
server_key: "4a9204bde6b63fea592acd4a51606d8f"
server_key_iv: "1d544cbe3e6164405e3093fe8e2a3c35"
server_key_version: "encryptionkey@0"
I20260811 17:14:21.788160 28632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "16e20e18f81b4dee8c466630c5e2bfbe"
format_stamp: "Formatted at 2026-08-11 17:14:21 on dist-test-slave-fd7l"
server_key: "4a9204bde6b63fea592acd4a51606d8f"
server_key_iv: "1d544cbe3e6164405e3093fe8e2a3c35"
server_key_version: "encryptionkey@0"
I20260811 17:14:21.792364 28632 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:14:21.795205 28648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:21.796860 28632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:21.797034 28632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "16e20e18f81b4dee8c466630c5e2bfbe"
format_stamp: "Formatted at 2026-08-11 17:14:21 on dist-test-slave-fd7l"
server_key: "4a9204bde6b63fea592acd4a51606d8f"
server_key_iv: "1d544cbe3e6164405e3093fe8e2a3c35"
server_key_version: "encryptionkey@0"
I20260811 17:14:21.797168 28632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:21.813663 28632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:21.817094 28632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:21.817266 28632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:21.817821 28632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:21.818656 28632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:21.818706 28632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:21.818745 28632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:21.818761 28632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:21.827705 28632 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:35843
I20260811 17:14:21.827725 28774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:35843 every 8 connection(s)
I20260811 17:14:21.828758 28632 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:14:21.837296 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 28632
I20260811 17:14:21.837453 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:14:21.837913 21722 external_mini_cluster.cc:1428] Setting key 60b82e97cc9c15c07300e7607b4a47a5
Aug 11 17:14:21 dist-test-slave-fd7l krb5kdc[28198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468461, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:21.844213 28292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:21.830757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:44893 (local address 127.21.54.190:39855)
0811 17:14:21.830907 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:21.830912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:21.831806 (+ 894us) server_negotiation.cc:407] Connection header received
0811 17:14:21.833375 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:21.833379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:21.833452 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:21.833565 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:21.834801 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.835364 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.836754 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.836982 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.837104 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:21.840271 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:21.840275 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:21.840278 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:21.840312 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:21.841943 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:21.842516 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:21.842523 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:21.842527 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:21.842627 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:21.843017 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:21.843020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:21.843022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:21.843222 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:21.843336 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:21.843900 (+ 564us) server_negotiation.cc:299] Negotiation successful
0811 17:14:21.844047 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:14:21.845187 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:21.831126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:39855 (local address 127.21.54.131:44893)
0811 17:14:21.831630 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:21.831677 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:21.833138 (+ 1461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:21.833750 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:21.833757 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:21.834149 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:21.834646 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:21.834659 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.835521 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.835525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:21.836608 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:21.836619 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.837165 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.837169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:21.837347 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:21.838131 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:21.838148 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:21.840080 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:21.842086 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:21.842092 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:21.842109 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:21.842399 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:21.842745 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:21.842748 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:21.842750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:21.842904 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:21.843351 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:21.843360 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:21.843649 (+ 289us) client_negotiation.cc:769] Sending connection context
0811 17:14:21.843858 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 17:14:21.844213 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":121,"threads_started":1}
I20260811 17:14:21.846393 28775 heartbeater.cc:347] Connected to a master server at 127.21.54.190:39855
I20260811 17:14:21.846632 28775 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:21.847116 28775 heartbeater.cc:511] Master 127.21.54.190:39855 requested a full tablet report, sending...
I20260811 17:14:21.848263 28248 ts_manager.cc:194] Registered new tserver with Master: 16e20e18f81b4dee8c466630c5e2bfbe (127.21.54.131:35843)
I20260811 17:14:21.848953 28248 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:44893
I20260811 17:14:21.852490 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/test-admin.keytab.
Aug 11 17:14:21 dist-test-slave-fd7l krb5kdc[28198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:21.874713 21722 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 17:14:21.877972 28783 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:21.876302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47304 (local address 127.21.54.129:44593)
0811 17:14:21.876724 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:21.876733 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:21.876759 (+ 26us) server_negotiation.cc:407] Connection header received
0811 17:14:21.876816 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:21.876821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:21.877015 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:21.877190 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:21.877783 (+ 593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47304: BlockingRecv error: recv got EOF from 127.0.0.1:47304 (error 108)
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":99,"threads_started":1}
W20260811 17:14:21.878142 21722 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/krb5cc' of type FILE
Aug 11 17:14:21 dist-test-slave-fd7l krb5kdc[28198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:21.879347 21722 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 17:14:21 dist-test-slave-fd7l krb5kdc[28198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:21.880466 21722 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 17:14:21 dist-test-slave-fd7l krb5kdc[28198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.129@KRBTEST.COM
I20260811 17:14:21.892366 28783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:21.881582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47308 (local address 127.21.54.129:44593)
0811 17:14:21.881759 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:21.881763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:21.881780 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:14:21.881893 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:21.881896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:21.881950 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:21.882034 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:21.882758 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.883690 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.884674 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.885136 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.885240 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:21.888204 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:21.888217 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:21.888233 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:21.888265 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:21.890200 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:21.890690 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:21.890696 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:21.890697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:21.890791 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:21.891105 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:21.891110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:21.891112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:21.891411 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:21.891630 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:21.891984 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 17:14:21.892217 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260811 17:14:21.897014 28783 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:21.895775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47324 (local address 127.21.54.129:44593)
0811 17:14:21.895930 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:21.895935 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:21.896058 (+ 123us) server_negotiation.cc:407] Connection header received
0811 17:14:21.896250 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:21.896252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:21.896295 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:21.896381 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:21.896890 (+ 509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47324: BlockingRecv error: recv got EOF from 127.0.0.1:47324 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 17:14:21.897426 21722 init.cc:244] Error while opening credential cache '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/krb5cc)
Aug 11 17:14:21 dist-test-slave-fd7l krb5kdc[28198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:21.898527 21722 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 17:14:21 dist-test-slave-fd7l krb5kdc[28198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:21.899446 21722 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 17:14:21 dist-test-slave-fd7l krb5kdc[28198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.129@KRBTEST.COM
I20260811 17:14:21.909665 28783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:21.900355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47328 (local address 127.21.54.129:44593)
0811 17:14:21.900470 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:21.900473 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:21.900573 (+ 100us) server_negotiation.cc:407] Connection header received
0811 17:14:21.900728 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:21.900730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:21.900771 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:21.900843 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:21.901600 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.902211 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.903019 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.903252 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.903331 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:21.905832 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:21.905835 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:21.905837 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:21.905889 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:21.907544 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:21.908120 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:21.908123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:21.908124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:21.908209 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:21.908664 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:21.908666 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:21.908667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:21.908847 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:21.908954 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:21.909288 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 17:14:21.909428 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 17:14:21.912900 28783 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:21.911650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47332 (local address 127.21.54.129:44593)
0811 17:14:21.911803 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:21.911809 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:21.911946 (+ 137us) server_negotiation.cc:407] Connection header received
0811 17:14:21.912106 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:21.912110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:21.912159 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:21.912247 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:21.912777 (+ 530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47332: BlockingRecv error: recv got EOF from 127.0.0.1:47332 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 17:14:21.913115 21722 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468428054096-21722-0/krb5kdc/krb5cc' of type FILE
Aug 11 17:14:21 dist-test-slave-fd7l krb5kdc[28198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:21.914268 21722 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 17:14:21 dist-test-slave-fd7l krb5kdc[28198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:21.915213 21722 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 17:14:21 dist-test-slave-fd7l krb5kdc[28198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.129@KRBTEST.COM
I20260811 17:14:21.925396 28783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:21.916119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47342 (local address 127.21.54.129:44593)
0811 17:14:21.916267 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:21.916270 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:21.916285 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:14:21.916430 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:21.916433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:21.916481 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:21.916645 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:21.917291 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.917846 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.918753 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:21.919011 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:21.919147 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:21.921709 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:21.921712 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:21.921715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:21.921750 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:21.923363 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:21.923898 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:21.923901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:21.923902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:21.924028 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:21.924315 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:21.924317 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:21.924318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:21.924574 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:21.924707 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:21.925053 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 17:14:21.925212 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 17:14:21.926765 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 28319
I20260811 17:14:21.936422 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 28477
I20260811 17:14:21.947283 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 28632
I20260811 17:14:21.957365 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 28211
2026-08-11T17:14:21Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2476 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:14:22 dist-test-slave-fd7l krb5kdc[28797](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:14:22 dist-test-slave-fd7l krb5kdc[28797](info): set up 2 sockets
Aug 11 17:14:22 dist-test-slave-fd7l krb5kdc[28797](info): commencing operation
krb5kdc: starting...
W20260811 17:14:23.102823 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:14:23 dist-test-slave-fd7l krb5kdc[28797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:14:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:14:23Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.190.keytab.
Entry for principal oryx/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.190.keytab.
I20260811 17:14:23.257392 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:38223
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.190.keytab
--principal=oryx/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:46121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:38223
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:23.384900 28812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:23.385286 28812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:23.385372 28812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:23.389750 28812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:23.389861 28812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:23.389901 28812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:23.389936 28812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:23.389968 28812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:14:23.395802 28812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:46121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:38223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.190.keytab
--principal=oryx/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:38223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:23.397121 28812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:23.398281 28812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:23.408079 28820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:23.408136 28818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:23.408079 28817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:23.408366 28812 server_base.cc:1034] running on GCE node
I20260811 17:14:23.409152 28812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:23.410501 28812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:23.411753 28812 hybrid_clock.cc:648] HybridClock initialized: now 1786468463411714 us; error 58 us; skew 500 ppm
Aug 11 17:14:23 dist-test-slave-fd7l krb5kdc[28797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468463, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:23.415109 28812 init.cc:377] Logged in from keytab as oryx/127.21.54.190@KRBTEST.COM (short username oryx)
I20260811 17:14:23.416131 28812 webserver.cc:460] Webserver started at http://127.21.54.190:34275/ using document root <none> and password file <none>
I20260811 17:14:23.416801 28812 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:23.416872 28812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:23.417063 28812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:23.419274 28812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "8aa7804a9c154946b9a8aad1210f7cf7"
format_stamp: "Formatted at 2026-08-11 17:14:23 on dist-test-slave-fd7l"
server_key: "1db7248c19cdba55cae01176d51d4d12"
server_key_iv: "68bdd9df93c577c69778fdc87c8d1834"
server_key_version: "encryptionkey@0"
I20260811 17:14:23.419862 28812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "8aa7804a9c154946b9a8aad1210f7cf7"
format_stamp: "Formatted at 2026-08-11 17:14:23 on dist-test-slave-fd7l"
server_key: "1db7248c19cdba55cae01176d51d4d12"
server_key_iv: "68bdd9df93c577c69778fdc87c8d1834"
server_key_version: "encryptionkey@0"
I20260811 17:14:23.424196 28812 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:14:23.427611 28827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:23.429401 28812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 17:14:23.429574 28812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "8aa7804a9c154946b9a8aad1210f7cf7"
format_stamp: "Formatted at 2026-08-11 17:14:23 on dist-test-slave-fd7l"
server_key: "1db7248c19cdba55cae01176d51d4d12"
server_key_iv: "68bdd9df93c577c69778fdc87c8d1834"
server_key_version: "encryptionkey@0"
I20260811 17:14:23.429729 28812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:23.454909 28812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:23.458781 28812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:23.459043 28812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:23.467687 28812 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:38223
I20260811 17:14:23.468065 28890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:38223 every 8 connection(s)
I20260811 17:14:23.469246 28812 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:14:23.472733 28892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:23.477193 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 28812
I20260811 17:14:23.477339 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:14:23.477671 21722 external_mini_cluster.cc:1428] Setting key 379d0ea633e7907fe0ca3b5cff376738
I20260811 17:14:23.481503 28892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7: Bootstrap starting.
I20260811 17:14:23.484364 28892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:23.485473 28892 log.cc:824] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:23.488047 28892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7: No bootstrap required, opened a new log
Aug 11 17:14:23 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM
I20260811 17:14:23.491981 28892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } }
I20260811 17:14:23.492362 28892 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:23.492444 28892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aa7804a9c154946b9a8aad1210f7cf7, State: Initialized, Role: FOLLOWER
I20260811 17:14:23.493104 28892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } }
I20260811 17:14:23.493325 28892 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:14:23.493431 28892 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:14:23.493557 28892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:23.494972 28892 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } }
I20260811 17:14:23.495399 28892 leader_election.cc:302] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8aa7804a9c154946b9a8aad1210f7cf7; no voters:
I20260811 17:14:23.495779 28892 leader_election.cc:288] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:14:23.495841 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:23.479381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50652 (local address 127.21.54.190:38223)
0811 17:14:23.479941 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:23.479970 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:23.480022 (+ 52us) server_negotiation.cc:407] Connection header received
0811 17:14:23.481000 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:23.481030 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:23.481544 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:23.481985 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:23.482937 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:23.484022 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:23.485278 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:23.485907 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:23.486099 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:23.489364 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:23.489370 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:23.489385 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:23.489418 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:23.491994 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:23.492673 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:23.492679 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:23.492685 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:23.492814 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:23.493166 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:23.493169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:23.493170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:23.493785 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:23.493948 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:23.494402 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 17:14:23.494782 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":181,"threads_started":1}
I20260811 17:14:23.495913 28900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:23.496356 28900 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [term 1 LEADER]: Becoming Leader. State: Replica: 8aa7804a9c154946b9a8aad1210f7cf7, State: Running, Role: LEADER
I20260811 17:14:23.496833 28900 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } }
I20260811 17:14:23.497380 28892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:23.499215 28902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } }
I20260811 17:14:23.499357 28902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:23.499779 28903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8aa7804a9c154946b9a8aad1210f7cf7. Latest consensus state: current_term: 1 leader_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } }
I20260811 17:14:23.499893 28912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:14:23.503523 28912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:14:23.504621 28903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:23.510741 28912 catalog_manager.cc:1347] Generated new cluster ID: 59c29e5bbd3b4e5ebe4dee805aa789d1
I20260811 17:14:23.510859 28912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:14:23.529269 28912 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:14:23.530853 28912 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:14:23.554417 28912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7: Generated new TSK 0
I20260811 17:14:23.555598 28912 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.129.keytab.
Entry for principal oryx/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.129.keytab.
I20260811 17:14:23.601708 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:38223
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.129.keytab
--principal=oryx/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:46121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:23.724309 28924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:23.724709 28924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:23.724828 28924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:23.728926 28924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:23.728995 28924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:23.729058 28924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:14:23.735260 28924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:46121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.129.keytab
--principal=oryx/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:38223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:23.736495 28924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:23.737637 28924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:23.749006 28930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:23.749135 28931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:23.749006 28935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:23.749383 28924 server_base.cc:1034] running on GCE node
I20260811 17:14:23.749938 28924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:23.750641 28924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:23.751865 28924 hybrid_clock.cc:648] HybridClock initialized: now 1786468463751843 us; error 32 us; skew 500 ppm
Aug 11 17:14:23 dist-test-slave-fd7l krb5kdc[28797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468463, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:23.755007 28924 init.cc:377] Logged in from keytab as oryx/127.21.54.129@KRBTEST.COM (short username oryx)
I20260811 17:14:23.755903 28924 webserver.cc:460] Webserver started at http://127.21.54.129:37823/ using document root <none> and password file <none>
I20260811 17:14:23.756479 28924 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:23.756604 28924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:23.756831 28924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:23.758970 28924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "af046522b683455f8989be6e327b0099"
format_stamp: "Formatted at 2026-08-11 17:14:23 on dist-test-slave-fd7l"
server_key: "7c0499ada3b8275dfc99e608fbf203b7"
server_key_iv: "7d9ee891db56f6b3cc153b80e95792ac"
server_key_version: "encryptionkey@0"
I20260811 17:14:23.759590 28924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "af046522b683455f8989be6e327b0099"
format_stamp: "Formatted at 2026-08-11 17:14:23 on dist-test-slave-fd7l"
server_key: "7c0499ada3b8275dfc99e608fbf203b7"
server_key_iv: "7d9ee891db56f6b3cc153b80e95792ac"
server_key_version: "encryptionkey@0"
I20260811 17:14:23.764130 28924 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 17:14:23.767165 28942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:23.768684 28924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:23.768828 28924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "af046522b683455f8989be6e327b0099"
format_stamp: "Formatted at 2026-08-11 17:14:23 on dist-test-slave-fd7l"
server_key: "7c0499ada3b8275dfc99e608fbf203b7"
server_key_iv: "7d9ee891db56f6b3cc153b80e95792ac"
server_key_version: "encryptionkey@0"
I20260811 17:14:23.768967 28924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:23.802057 28924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:23.806253 28924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:23.806494 28924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:23.807192 28924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:23.808179 28924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:23.808279 28924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:23.808354 28924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:23.808401 28924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:23.819288 28924 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:37479
I20260811 17:14:23.819309 29077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:37479 every 8 connection(s)
I20260811 17:14:23.820360 28924 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:14:23.821197 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 28924
I20260811 17:14:23.821327 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:14:23.821655 21722 external_mini_cluster.cc:1428] Setting key 562eb38789920d77d6b3cc22d1d8299d
Aug 11 17:14:23 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468463, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.21.54.130@KRBTEST.COM; defaulting to no policy
I20260811 17:14:23.836274 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:23.822602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:57039 (local address 127.21.54.190:38223)
0811 17:14:23.822755 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:23.822758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:23.823902 (+ 1144us) server_negotiation.cc:407] Connection header received
0811 17:14:23.825545 (+ 1643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:23.825549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:23.825609 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:23.825691 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:23.826976 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:23.827538 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:23.828808 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:23.829058 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:23.829179 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:23.832124 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:23.832128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:23.832130 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:23.832167 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:23.833902 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:23.834521 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:23.834525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:23.834527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:23.834620 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:23.835018 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:23.835022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:23.835023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:23.835230 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:23.835340 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:23.835894 (+ 554us) server_negotiation.cc:299] Negotiation successful
0811 17:14:23.836041 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.54.130@KRBTEST.COM" created.
I20260811 17:14:23.837388 29080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:23.823214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.129:57039)
0811 17:14:23.823755 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:23.823801 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:23.825304 (+ 1503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:23.825875 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:23.825883 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:23.826278 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:23.826806 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:23.826820 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:23.827690 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:23.827694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:23.828679 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:23.828689 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:23.829227 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:23.829231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:23.829420 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:23.830088 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:23.830107 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:23.831922 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:23.834079 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:23.834086 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:23.834107 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:23.834388 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:23.834717 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:23.834720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:23.834723 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:23.834925 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:23.835341 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:23.835350 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:23.835643 (+ 293us) client_negotiation.cc:769] Sending connection context
0811 17:14:23.835854 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 17:14:23.836231 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":163,"threads_started":1}
I20260811 17:14:23.839246 29078 heartbeater.cc:347] Connected to a master server at 127.21.54.190:38223
I20260811 17:14:23.839566 29078 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:23.840148 29078 heartbeater.cc:511] Master 127.21.54.190:38223 requested a full tablet report, sending...
I20260811 17:14:23.842082 28845 ts_manager.cc:194] Registered new tserver with Master: af046522b683455f8989be6e327b0099 (127.21.54.129:37479)
I20260811 17:14:23.843511 28845 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:57039
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.130.keytab.
Entry for principal oryx/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.130.keytab.
I20260811 17:14:23.856415 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:38223
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.130.keytab
--principal=oryx/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:46121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:23.980706 29084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:23.981016 29084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:23.981109 29084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:23.985324 29084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:23.985412 29084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:23.985512 29084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:14:23.991140 29084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:46121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.130.keytab
--principal=oryx/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:38223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:23.992305 29084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:23.993434 29084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:24.004431 29089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:24.004432 29090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:24.004882 29092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:24.004948 29084 server_base.cc:1034] running on GCE node
I20260811 17:14:24.005533 29084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:24.006455 29084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:24.007666 29084 hybrid_clock.cc:648] HybridClock initialized: now 1786468464007644 us; error 30 us; skew 500 ppm
Aug 11 17:14:24 dist-test-slave-fd7l krb5kdc[28797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468464, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:24.011565 29084 init.cc:377] Logged in from keytab as oryx/127.21.54.130@KRBTEST.COM (short username oryx)
I20260811 17:14:24.012676 29084 webserver.cc:460] Webserver started at http://127.21.54.130:43139/ using document root <none> and password file <none>
I20260811 17:14:24.013295 29084 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:24.013361 29084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:24.013598 29084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:24.015826 29084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "6127ec870149479e80c2b8d80e1e20a1"
format_stamp: "Formatted at 2026-08-11 17:14:24 on dist-test-slave-fd7l"
server_key: "515e621984c45b53eff8f2f2ef90ec6b"
server_key_iv: "ca9aa4fb70c10d39f9144f83678cea3a"
server_key_version: "encryptionkey@0"
I20260811 17:14:24.016417 29084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "6127ec870149479e80c2b8d80e1e20a1"
format_stamp: "Formatted at 2026-08-11 17:14:24 on dist-test-slave-fd7l"
server_key: "515e621984c45b53eff8f2f2ef90ec6b"
server_key_iv: "ca9aa4fb70c10d39f9144f83678cea3a"
server_key_version: "encryptionkey@0"
I20260811 17:14:24.020990 29084 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:14:24.024132 29100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:24.025732 29084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:24.025902 29084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "6127ec870149479e80c2b8d80e1e20a1"
format_stamp: "Formatted at 2026-08-11 17:14:24 on dist-test-slave-fd7l"
server_key: "515e621984c45b53eff8f2f2ef90ec6b"
server_key_iv: "ca9aa4fb70c10d39f9144f83678cea3a"
server_key_version: "encryptionkey@0"
I20260811 17:14:24.026055 29084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:24.046520 29084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:24.050598 29084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:24.050868 29084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:24.051605 29084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:24.052717 29084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:24.052796 29084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:24.052876 29084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:24.052940 29084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:24.065415 29084 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:42439
I20260811 17:14:24.065433 29238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:42439 every 8 connection(s)
I20260811 17:14:24.066613 29084 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:14:24.069626 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 29084
I20260811 17:14:24.069778 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:14:24.070196 21722 external_mini_cluster.cc:1428] Setting key 7b744833aeee7179c5d2d8d8c5bac641
Aug 11 17:14:24 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468464, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.21.54.131@KRBTEST.COM; defaulting to no policy
I20260811 17:14:24.084698 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.069161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:44963 (local address 127.21.54.190:38223)
0811 17:14:24.069327 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:24.069331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:24.070663 (+ 1332us) server_negotiation.cc:407] Connection header received
0811 17:14:24.072735 (+ 2072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:24.072739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:24.072824 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:24.072892 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:24.075030 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.075633 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.077064 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.077325 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.077457 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.080388 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:24.080392 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:24.080394 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:24.080421 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:24.082168 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.082759 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.082764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.082766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.082864 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.083250 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.083255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.083257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.083468 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:24.083583 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:24.084168 (+ 585us) server_negotiation.cc:299] Negotiation successful
0811 17:14:24.084408 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 17:14:24.085680 29241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.069665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.130:44963)
0811 17:14:24.070459 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:24.070531 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:24.072423 (+ 1892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:24.073393 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:24.073407 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:24.074023 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:24.074857 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.074884 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.075789 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.075792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.076908 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.076919 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.077506 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.077512 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.077713 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.078411 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:24.078431 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:24.080184 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:24.082304 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.082311 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.082330 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.082639 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.082966 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.082971 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.082973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.083140 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.083605 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:24.083613 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:24.083894 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 17:14:24.084117 (+ 223us) client_negotiation.cc:240] Negotiation successful
0811 17:14:24.084506 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":177,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.54.131@KRBTEST.COM" created.
I20260811 17:14:24.087322 29239 heartbeater.cc:347] Connected to a master server at 127.21.54.190:38223
I20260811 17:14:24.087622 29239 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:24.088336 29239 heartbeater.cc:511] Master 127.21.54.190:38223 requested a full tablet report, sending...
I20260811 17:14:24.089740 28845 ts_manager.cc:194] Registered new tserver with Master: 6127ec870149479e80c2b8d80e1e20a1 (127.21.54.130:42439)
I20260811 17:14:24.090430 28845 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:44963
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.131.keytab.
Entry for principal oryx/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.131.keytab.
I20260811 17:14:24.108259 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:38223
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.131.keytab
--principal=oryx/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:46121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:24.227080 29246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:24.227387 29246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:24.227469 29246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:24.232024 29246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:24.232101 29246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:24.232165 29246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:14:24.237977 29246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:46121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.131.keytab
--principal=oryx/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:38223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:24.239159 29246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:24.240257 29246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:24.250707 29255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:24.250708 29251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:24.250944 29246 server_base.cc:1034] running on GCE node
W20260811 17:14:24.250707 29252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:24.251654 29246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:24.252375 29246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:24.253545 29246 hybrid_clock.cc:648] HybridClock initialized: now 1786468464253519 us; error 38 us; skew 500 ppm
Aug 11 17:14:24 dist-test-slave-fd7l krb5kdc[28797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468464, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:24.256857 29246 init.cc:377] Logged in from keytab as oryx/127.21.54.131@KRBTEST.COM (short username oryx)
I20260811 17:14:24.257902 29246 webserver.cc:460] Webserver started at http://127.21.54.131:38541/ using document root <none> and password file <none>
I20260811 17:14:24.258529 29246 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:24.258599 29246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:24.258837 29246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:24.261171 29246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "23f3969ead3b4e70b1014825b3a82790"
format_stamp: "Formatted at 2026-08-11 17:14:24 on dist-test-slave-fd7l"
server_key: "e1ea614cff37f921a9fea76b923c2f5a"
server_key_iv: "166d8d2afdb31c6d74c9c2ff4a9d9132"
server_key_version: "encryptionkey@0"
I20260811 17:14:24.261792 29246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "23f3969ead3b4e70b1014825b3a82790"
format_stamp: "Formatted at 2026-08-11 17:14:24 on dist-test-slave-fd7l"
server_key: "e1ea614cff37f921a9fea76b923c2f5a"
server_key_iv: "166d8d2afdb31c6d74c9c2ff4a9d9132"
server_key_version: "encryptionkey@0"
I20260811 17:14:24.266060 29246 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:14:24.269219 29262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:24.270675 29246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:14:24.270828 29246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "23f3969ead3b4e70b1014825b3a82790"
format_stamp: "Formatted at 2026-08-11 17:14:24 on dist-test-slave-fd7l"
server_key: "e1ea614cff37f921a9fea76b923c2f5a"
server_key_iv: "166d8d2afdb31c6d74c9c2ff4a9d9132"
server_key_version: "encryptionkey@0"
I20260811 17:14:24.270960 29246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:24.287747 29246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:24.291363 29246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:24.291633 29246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:24.292307 29246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:24.293252 29246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:24.293334 29246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:24.293444 29246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:24.293488 29246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:24.303349 29246 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:41509
I20260811 17:14:24.303372 29397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:41509 every 8 connection(s)
I20260811 17:14:24.304322 29246 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:14:24.307303 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 29246
I20260811 17:14:24.307464 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:14:24.307845 21722 external_mini_cluster.cc:1428] Setting key cbc04b66d51dd30b83d48d41b8160570
Aug 11 17:14:24 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468464, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM
I20260811 17:14:24.320449 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.306311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:50835 (local address 127.21.54.190:38223)
0811 17:14:24.306493 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:24.306497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:24.307554 (+ 1057us) server_negotiation.cc:407] Connection header received
0811 17:14:24.309304 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:24.309307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:24.309384 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:24.309469 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:24.310884 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.311464 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.312819 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.313073 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.313208 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.316083 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:24.316087 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:24.316089 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:24.316119 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:24.318034 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.318680 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.318683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.318685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.318784 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.319261 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.319264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.319266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.319499 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:24.319590 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:24.320185 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 17:14:24.320308 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 17:14:24.321746 29402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.306697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.131:50835)
0811 17:14:24.307287 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:24.307352 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:24.309024 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:24.309649 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:24.309656 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:24.310132 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:24.310709 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.310724 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.311632 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.311635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.312669 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.312681 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.313263 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.313268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.313470 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.314128 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:24.314146 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:24.315875 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:24.318194 (+ 2319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.318201 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.318217 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.318547 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.318910 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.318913 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.318914 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.319127 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.319613 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:24.319619 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:24.319895 (+ 276us) client_negotiation.cc:769] Sending connection context
0811 17:14:24.320107 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 17:14:24.320466 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":118,"threads_started":1}
I20260811 17:14:24.322994 29399 heartbeater.cc:347] Connected to a master server at 127.21.54.190:38223
I20260811 17:14:24.323275 29399 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:24.323850 29399 heartbeater.cc:511] Master 127.21.54.190:38223 requested a full tablet report, sending...
I20260811 17:14:24.325069 28847 ts_manager.cc:194] Registered new tserver with Master: 23f3969ead3b4e70b1014825b3a82790 (127.21.54.131:41509)
I20260811 17:14:24.325716 28847 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:50835
I20260811 17:14:24.334956 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.54.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.54.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.54.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.189.keytab.
Entry for principal oryx/127.21.54.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.189.keytab.
I20260811 17:14:24.368768 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.189:42221
--webserver_interface=127.21.54.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.189.keytab
--principal=oryx/127.21.54.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:46121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:38223,127.21.54.189:42221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:24.501813 29406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:24.502245 29406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:24.502374 29406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:24.507403 29406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:24.507511 29406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:24.507556 29406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:24.507627 29406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:24.507668 29406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:14:24.513851 29406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:46121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:38223,127.21.54.189:42221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.189.keytab
--principal=oryx/127.21.54.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.189:42221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.54.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
W20260811 17:14:24.514166 29406 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.21.54.190:38223,127.21.54.189:42221'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 17:14:24.515158 29406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:24.516256 29406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:24.525902 29412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:24.526072 29406 server_base.cc:1034] running on GCE node
W20260811 17:14:24.525979 29411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:24.526330 29415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:24.526968 29406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:24.528278 29406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:24.529469 29406 hybrid_clock.cc:648] HybridClock initialized: now 1786468464529456 us; error 42 us; skew 500 ppm
Aug 11 17:14:24 dist-test-slave-fd7l krb5kdc[28797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468464, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:24.533231 29406 init.cc:377] Logged in from keytab as oryx/127.21.54.189@KRBTEST.COM (short username oryx)
I20260811 17:14:24.534512 29406 webserver.cc:460] Webserver started at http://127.21.54.189:44085/ using document root <none> and password file <none>
I20260811 17:14:24.535197 29406 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:24.535297 29406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:24.535550 29406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:24.537878 29406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/data/instance:
uuid: "e72959b06b32490fb14d04885db83c44"
format_stamp: "Formatted at 2026-08-11 17:14:24 on dist-test-slave-fd7l"
server_key: "545a2412ae28b49b2fb63b3e34bb80b7"
server_key_iv: "82cdb8c9014d3c0e4593ff11f9faf964"
server_key_version: "encryptionkey@0"
I20260811 17:14:24.538493 29406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/wal/instance:
uuid: "e72959b06b32490fb14d04885db83c44"
format_stamp: "Formatted at 2026-08-11 17:14:24 on dist-test-slave-fd7l"
server_key: "545a2412ae28b49b2fb63b3e34bb80b7"
server_key_iv: "82cdb8c9014d3c0e4593ff11f9faf964"
server_key_version: "encryptionkey@0"
I20260811 17:14:24.542814 29406 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:14:24.545578 29422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:24.546900 29406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:24.547067 29406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/wal
uuid: "e72959b06b32490fb14d04885db83c44"
format_stamp: "Formatted at 2026-08-11 17:14:24 on dist-test-slave-fd7l"
server_key: "545a2412ae28b49b2fb63b3e34bb80b7"
server_key_iv: "82cdb8c9014d3c0e4593ff11f9faf964"
server_key_version: "encryptionkey@0"
I20260811 17:14:24.547204 29406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:24.585333 29406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:24.589334 29406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:24.589588 29406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:24.596875 29406 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.189:42221
I20260811 17:14:24.597188 29482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.189:42221 every 8 connection(s)
I20260811 17:14:24.598088 29406 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/data/info.pb
I20260811 17:14:24.601387 29483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:24.605394 29483 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:14:24.608189 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 29406
Aug 11 17:14:24 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468464, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.189@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM
Aug 11 17:14:24 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.54.189@KRBTEST.COM
I20260811 17:14:24.620311 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.607388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50654 (local address 127.21.54.190:38223)
0811 17:14:24.607553 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:24.607558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:24.608544 (+ 986us) server_negotiation.cc:407] Connection header received
0811 17:14:24.609659 (+ 1115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:24.609663 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:24.609725 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:24.609833 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:24.611019 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.611555 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.612824 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.613049 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.613164 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.616209 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:24.616213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:24.616215 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:24.616243 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:24.618060 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.618610 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.618614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.618616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.618707 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.619084 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.619089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.619091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.619331 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:24.619445 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:24.619999 (+ 554us) server_negotiation.cc:299] Negotiation successful
0811 17:14:24.620130 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 17:14:24.621217 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.609264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39648 (local address 127.21.54.189:42221)
0811 17:14:24.609679 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:24.609693 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:24.609740 (+ 47us) server_negotiation.cc:407] Connection header received
0811 17:14:24.609961 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:24.609966 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:24.610156 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:24.610352 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:24.611235 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.612236 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.613228 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.613625 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.613784 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.616766 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:24.616773 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:24.616785 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:24.616809 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:24.618827 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.619365 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.619376 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.619378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.619461 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.619795 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.619798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.619800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.620057 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:24.620287 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:24.620910 (+ 623us) server_negotiation.cc:299] Negotiation successful
0811 17:14:24.621060 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":296,"spinlock_wait_cycles":1280,"thread_start_us":148,"threads_started":1}
I20260811 17:14:24.621222 29485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.607818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:50654)
0811 17:14:24.608334 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:24.608389 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:24.609398 (+ 1009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:24.609995 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:24.610003 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:24.610344 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:24.610869 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.610883 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.611706 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.611709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.612698 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.612707 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.613207 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.613211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.613431 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.614128 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:24.614152 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:24.616043 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:24.618188 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.618199 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.618211 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.618499 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.618809 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.618812 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.618814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.618982 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.619445 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:24.619453 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:24.619740 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 17:14:24.619943 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 17:14:24.620281 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":173,"thread_start_us":155,"threads_started":1}
I20260811 17:14:24.622829 29483 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:14:24 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468464, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.189@KRBTEST.COM for oryx/127.21.54.189@KRBTEST.COM
I20260811 17:14:24.632555 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.623626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39650 (local address 127.21.54.189:42221)
0811 17:14:24.623726 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:24.623729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:24.623770 (+ 41us) server_negotiation.cc:407] Connection header received
0811 17:14:24.623888 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:24.623891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:24.623930 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:24.623995 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:24.624416 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.625025 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.625861 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.626758 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.626858 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.629223 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:24.629226 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:24.629229 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:24.629252 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:24.630821 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.631326 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.631332 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.631334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.631411 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.631694 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.631698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.631700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.631889 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:24.631998 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:24.632239 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 17:14:24.632402 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":36}
I20260811 17:14:24.632555 29485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.623466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:42221 (local address 127.0.0.1:39650)
0811 17:14:24.623716 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:24.623730 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:24.623809 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:24.624026 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:24.624028 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:24.624208 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:24.624309 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.624314 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.625141 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.625144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.625730 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.625739 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.626898 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.626901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.626993 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.627464 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:24.627479 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:24.629054 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:24.631032 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.631034 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.631036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.631225 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.631520 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.631523 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.631525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.631604 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.631993 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:24.631996 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:24.632087 (+ 91us) client_negotiation.cc:769] Sending connection context
0811 17:14:24.632195 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 17:14:24.632344 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260811 17:14:24.636026 29483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: Bootstrap starting.
I20260811 17:14:24.638682 29483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:24.639489 29483 log.cc:824] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:24.641891 29483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: No bootstrap required, opened a new log
I20260811 17:14:24.645187 29483 raft_consensus.cc:348] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } }
I20260811 17:14:24.645433 29483 raft_consensus.cc:374] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:24.645478 29483 raft_consensus.cc:729] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72959b06b32490fb14d04885db83c44, State: Initialized, Role: FOLLOWER
I20260811 17:14:24.645915 29483 consensus_queue.cc:260] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } }
I20260811 17:14:24.646446 29490 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } } }
I20260811 17:14:24.646598 29490 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:14:24.647284 29483 sys_catalog.cc:564] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:24.651855 29490 raft_consensus.cc:482] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:14:24.652074 29490 raft_consensus.cc:504] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } }
I20260811 17:14:24.653003 29490 leader_election.cc:288] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8aa7804a9c154946b9a8aad1210f7cf7 (127.21.54.190:38223)
W20260811 17:14:24.653149 29503 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:14:24.653244 29503 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:14:24.654076 28861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e72959b06b32490fb14d04885db83c44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" is_pre_election: true
I20260811 17:14:24.654783 29426 leader_election.cc:302] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: e72959b06b32490fb14d04885db83c44; no voters: 8aa7804a9c154946b9a8aad1210f7cf7
I20260811 17:14:24.655094 29490 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 17:14:24.656178 29406 master_runner.cc:428] Detected that this master e72959b06b32490fb14d04885db83c44 is joining an existing cluster
I20260811 17:14:24.656245 29406 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.54.189:42221
I20260811 17:14:24.669598 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.658943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50668 (local address 127.21.54.190:38223)
0811 17:14:24.659169 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:24.659173 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:24.659359 (+ 186us) server_negotiation.cc:407] Connection header received
0811 17:14:24.659530 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:24.659536 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:24.659601 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:24.659712 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:24.660719 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.661617 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.662718 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.662997 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.663138 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.664734 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:24.664741 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:24.664745 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:24.664788 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:24.667204 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.667953 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.667959 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.667961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.668084 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.668466 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.668470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.668472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.668772 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:24.668908 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:24.669257 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 17:14:24.669414 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260811 17:14:24.669920 29512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.659748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:42221 (local address 127.0.0.1:39660)
0811 17:14:24.660098 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:24.660115 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:24.660216 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:24.660499 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:24.660502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:24.660757 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:24.660887 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.660895 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.661929 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.661933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.663451 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.663463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.664139 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.664143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.664277 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.664894 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:24.664909 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:24.665326 (+ 417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:24.668138 (+ 2812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.668141 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.668143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.668388 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.668705 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.668708 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.668710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.668784 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.669219 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:24.669222 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:24.669301 (+ 79us) client_negotiation.cc:769] Sending connection context
0811 17:14:24.669641 (+ 340us) client_negotiation.cc:240] Negotiation successful
0811 17:14:24.669749 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":4,"thread_start_us":120,"threads_started":1}
I20260811 17:14:24.670081 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.659596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39660 (local address 127.21.54.189:42221)
0811 17:14:24.659735 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:24.659739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:24.660213 (+ 474us) server_negotiation.cc:407] Connection header received
0811 17:14:24.660293 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:24.660296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:24.660354 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:24.660470 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:24.661026 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.661798 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.663606 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.663979 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.664136 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.665491 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:24.665499 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:24.665506 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:24.665544 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:24.668016 (+ 2472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.668497 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.668500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.668502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.668606 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.668867 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.668870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.668872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.669091 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:24.669162 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:24.669446 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 17:14:24.669562 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":64,"spinlock_wait_cycles":2432}
I20260811 17:14:24.669620 29511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.658849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:50668)
0811 17:14:24.659229 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:24.659246 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:24.659370 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:24.659952 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:24.659956 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:24.660286 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:24.660490 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.660499 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.661781 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.661785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.662560 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.662570 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.663161 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.663166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.663277 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.663964 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:24.663985 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:24.664481 (+ 496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:24.667343 (+ 2862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.667347 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.667349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.667822 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.668230 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.668233 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.668234 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.668331 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.668901 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:24.668905 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:24.669031 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 17:14:24.669177 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 17:14:24.669367 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"spinlock_wait_cycles":896,"thread_start_us":151,"threads_started":1}
Aug 11 17:14:24 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468463, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.190@KRBTEST.COM for oryx/127.21.54.189@KRBTEST.COM
WARNING: no policy specified for oryx/127.21.54.188@KRBTEST.COM; defaulting to no policy
I20260811 17:14:24.687297 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.677143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39668 (local address 127.21.54.189:42221)
0811 17:14:24.677265 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:24.677270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:24.677284 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:14:24.677354 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:24.677357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:24.677409 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:24.677499 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:24.678203 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.678723 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.679644 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.679866 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.679977 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.683021 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:24.683025 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:24.683028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:24.683059 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:24.684963 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.685606 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.685613 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.685617 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.685723 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.686066 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.686072 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.686074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.686359 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:24.686482 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:24.686934 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 17:14:24.687077 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 17:14:24.687647 29513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.676387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:42221 (local address 127.0.0.1:39668)
0811 17:14:24.676834 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:24.676902 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:24.677060 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:24.677560 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:24.677566 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:24.677932 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:24.678075 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.678082 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.678842 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.678847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.679517 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.679527 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.679985 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.679988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.680129 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.680774 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:24.680790 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:24.682850 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:24.685109 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.685121 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.685129 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.685464 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.685839 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.685842 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.685844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.685945 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.686502 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:24.686511 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:24.686668 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 17:14:24.686838 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 17:14:24.687076 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":119,"threads_started":1}
I20260811 17:14:24.689702 28847 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.54.189:42221: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e72959b06b32490fb14d04885db83c44"
member_type: NON_VOTER
last_known_addr {
host: "127.21.54.189"
port: 42221
}
attrs {
promote: true
}
}
dest_uuid: "8aa7804a9c154946b9a8aad1210f7cf7"
cas_config_opid_index: -1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.54.188@KRBTEST.COM" created.
I20260811 17:14:24.690771 28847 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: NON_VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: true } }
I20260811 17:14:24.693359 29516 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER e72959b06b32490fb14d04885db83c44 (127.21.54.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: NON_VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: true } } }
I20260811 17:14:24.694300 29454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:24.695739 29454 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 1 FOLLOWER]: Refusing update from remote peer 8aa7804a9c154946b9a8aad1210f7cf7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 17:14:24.697954 29516 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.54.189:42221
I20260811 17:14:24.698282 29516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: NON_VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: true } } }
I20260811 17:14:24.698439 29516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:24.698817 29406 master.cc:537] Master@127.21.54.189:42221 shutting down...
I20260811 17:14:24.699018 29519 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: NON_VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.002s
I20260811 17:14:24.704445 29406 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:24.705094 29406 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:24.705291 29406 tablet_replica.cc:330] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: stopping tablet replica
W20260811 17:14:24.705513 28835 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 17:14:24.706861 28835 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 -> Peer e72959b06b32490fb14d04885db83c44 (127.21.54.189:42221): Couldn't send request to peer e72959b06b32490fb14d04885db83c44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.54.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.188.keytab.
Entry for principal oryx/127.21.54.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.188.keytab.
I20260811 17:14:24.712581 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.188:44127
--webserver_interface=127.21.54.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.188.keytab
--principal=oryx/127.21.54.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:46121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:38223,127.21.54.189:42221,127.21.54.188:44127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:14:24.721444 29406 master.cc:552] Master@127.21.54.189:42221 shutdown complete.
I20260811 17:14:24.721532 29406 master_runner.cc:305] Clearing existing system tablet
I20260811 17:14:24.721800 29406 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:24.721861 29406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:24.723380 29406 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 17:14:24.724383 29527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:24.724713 29406 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:24.724848 29406 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 17:14:24.724882 29406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/wal
uuid: "e72959b06b32490fb14d04885db83c44"
format_stamp: "Formatted at 2026-08-11 17:14:24 on dist-test-slave-fd7l"
server_key: "545a2412ae28b49b2fb63b3e34bb80b7"
server_key_iv: "82cdb8c9014d3c0e4593ff11f9faf964"
server_key_version: "encryptionkey@0"
I20260811 17:14:24.727197 29406 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:24.729883 29406 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 17:14:24.729984 29406 log.cc:1196] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 17:14:24.730345 29406 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: Deleting consensus metadata
I20260811 17:14:24.730541 29406 master_runner.cc:315] Copying system tablet from 127.21.54.190:38223
I20260811 17:14:24.732026 29406 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: tablet copy: Beginning tablet copy session from remote peer at address 127.21.54.190:38223
I20260811 17:14:24.741765 29538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.732741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:50682)
0811 17:14:24.733068 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:24.733083 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:24.733218 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:24.733533 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:24.733536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:24.733775 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:24.733910 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.733918 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.735060 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.735063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.735958 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.735970 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.736597 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.736601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.736707 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.737336 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:24.737351 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:24.737789 (+ 438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:24.739902 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.739906 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.739908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.740192 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.740575 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.740578 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.740579 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.740684 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.741163 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:24.741166 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:24.741277 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 17:14:24.741389 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 17:14:24.741548 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":96,"threads_started":1}
I20260811 17:14:24.741779 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.732857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50682 (local address 127.21.54.190:38223)
0811 17:14:24.732995 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:24.733000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:24.733188 (+ 188us) server_negotiation.cc:407] Connection header received
0811 17:14:24.733305 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:24.733308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:24.733370 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:24.733466 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:24.734068 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.734937 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.736099 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.736447 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.736599 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.737949 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:24.737953 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:24.737956 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:24.737986 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:24.739771 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.740312 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.740316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.740318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.740427 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.740787 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.740791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.740793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.741054 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:24.741162 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:24.741471 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 17:14:24.741612 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:14:24.742990 28864 tablet_copy_service.cc:140] P 8aa7804a9c154946b9a8aad1210f7cf7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e72959b06b32490fb14d04885db83c44 ({username='oryx', principal='oryx/127.21.54.189@KRBTEST.COM'} at 127.0.0.1:50682)
I20260811 17:14:24.743175 28864 tablet_copy_service.cc:161] P 8aa7804a9c154946b9a8aad1210f7cf7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e72959b06b32490fb14d04885db83c44 at {username='oryx', principal='oryx/127.21.54.189@KRBTEST.COM'} at 127.0.0.1:50682: session id = e72959b06b32490fb14d04885db83c44-00000000000000000000000000000000
I20260811 17:14:24.745224 28864 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 17:14:24.747292 29406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 17:14:24.750065 29406 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: tablet copy: Starting download of 0 data blocks...
I20260811 17:14:24.750291 29406 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: tablet copy: Starting download of 1 WAL segments...
I20260811 17:14:24.752372 29406 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 17:14:24.754462 28864 tablet_copy_service.cc:342] P 8aa7804a9c154946b9a8aad1210f7cf7: Request end of tablet copy session e72959b06b32490fb14d04885db83c44-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.54.189@KRBTEST.COM'} at 127.0.0.1:50682
I20260811 17:14:24.754592 28864 tablet_copy_service.cc:433] P 8aa7804a9c154946b9a8aad1210f7cf7: ending tablet copy session e72959b06b32490fb14d04885db83c44-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e72959b06b32490fb14d04885db83c44
I20260811 17:14:24.763367 29406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:24.765404 29539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:24.765479 29542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:24.767946 29540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:24.768703 29406 server_base.cc:1034] running on GCE node
I20260811 17:14:24.768893 29406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:24.769279 29406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:24.770447 29406 hybrid_clock.cc:648] HybridClock initialized: now 1786468464770416 us; error 47 us; skew 500 ppm
Aug 11 17:14:24 dist-test-slave-fd7l krb5kdc[28797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468464, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:24.772305 29406 init.cc:377] Logged in from keytab as oryx/127.21.54.189@KRBTEST.COM (short username oryx)
I20260811 17:14:24.772923 29406 webserver.cc:460] Webserver started at http://127.21.54.189:41259/ using document root <none> and password file <none>
I20260811 17:14:24.773128 29406 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:24.773170 29406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:24.774605 29406 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 17:14:24.775748 29550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:24.775997 29406 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 17:14:24.776119 29406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/wal
uuid: "e72959b06b32490fb14d04885db83c44"
format_stamp: "Formatted at 2026-08-11 17:14:24 on dist-test-slave-fd7l"
server_key: "545a2412ae28b49b2fb63b3e34bb80b7"
server_key_iv: "82cdb8c9014d3c0e4593ff11f9faf964"
server_key_version: "encryptionkey@0"
I20260811 17:14:24.776273 29406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:24.808852 29406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:24.811416 29406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:24.817071 29406 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.189:42221
I20260811 17:14:24.817883 29406 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-1/data/info.pb
I20260811 17:14:24.818609 29612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.189:42221 every 8 connection(s)
I20260811 17:14:24.820623 29613 sys_catalog.cc:263] Verifying existing consensus state
I20260811 17:14:24.821894 29613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: Bootstrap starting.
I20260811 17:14:24.833745 29613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:14:24.834327 29613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: Bootstrap complete.
I20260811 17:14:24.835155 29613 raft_consensus.cc:348] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: NON_VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: true } }
I20260811 17:14:24.835405 29613 raft_consensus.cc:729] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e72959b06b32490fb14d04885db83c44, State: Initialized, Role: LEARNER
I20260811 17:14:24.835577 29613 consensus_queue.cc:260] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: NON_VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: true } }
I20260811 17:14:24.836052 29619 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: NON_VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: true } } }
I20260811 17:14:24.836160 29613 sys_catalog.cc:564] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:24.836190 29619 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [sys.catalog]: This master's current role is: LEARNER
I20260811 17:14:24.839479 29633 catalog_manager.cc:1259] Loaded cluster ID: 59c29e5bbd3b4e5ebe4dee805aa789d1
I20260811 17:14:24.839555 29633 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: loading cluster ID for follower catalog manager: success
I20260811 17:14:24.841936 29633 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: acquiring CA information for follower catalog manager: success
I20260811 17:14:24.843798 29633 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 17:14:24.846110 29078 heartbeater.cc:503] Master 127.21.54.190:38223 was elected leader, sending a full tablet report...
W20260811 17:14:24.856971 29524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:24.857342 29524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:24.857425 29524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:24.861763 29524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:24.861871 29524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:24.861907 29524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:24.861948 29524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:24.861984 29524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:14:24.867786 29524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:46121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:38223,127.21.54.189:42221,127.21.54.188:44127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.188.keytab
--principal=oryx/127.21.54.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.188:44127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.54.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:24.869170 29524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:24.870322 29524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:24.880647 29639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:24.880599 29641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:24.880566 29638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:24.881044 29524 server_base.cc:1034] running on GCE node
I20260811 17:14:24.881717 29524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:24.883037 29524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:24.884265 29524 hybrid_clock.cc:648] HybridClock initialized: now 1786468464884242 us; error 41 us; skew 500 ppm
Aug 11 17:14:24 dist-test-slave-fd7l krb5kdc[28797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468464, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:24.887745 29524 init.cc:377] Logged in from keytab as oryx/127.21.54.188@KRBTEST.COM (short username oryx)
I20260811 17:14:24.888942 29524 webserver.cc:460] Webserver started at http://127.21.54.188:43707/ using document root <none> and password file <none>
I20260811 17:14:24.889575 29524 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:24.889673 29524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:24.889927 29524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:24.892201 29524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/data/instance:
uuid: "0613d8d679e648219d0996a3849a3668"
format_stamp: "Formatted at 2026-08-11 17:14:24 on dist-test-slave-fd7l"
server_key: "c9254383b116756090f7db02a137b788"
server_key_iv: "3742a99f8ecdcd2e59149582a6f57eb6"
server_key_version: "encryptionkey@0"
I20260811 17:14:24.892855 29524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/wal/instance:
uuid: "0613d8d679e648219d0996a3849a3668"
format_stamp: "Formatted at 2026-08-11 17:14:24 on dist-test-slave-fd7l"
server_key: "c9254383b116756090f7db02a137b788"
server_key_iv: "3742a99f8ecdcd2e59149582a6f57eb6"
server_key_version: "encryptionkey@0"
I20260811 17:14:24.897030 29524 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:14:24.900164 29649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:24.901714 29524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:24.901875 29524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/wal
uuid: "0613d8d679e648219d0996a3849a3668"
format_stamp: "Formatted at 2026-08-11 17:14:24 on dist-test-slave-fd7l"
server_key: "c9254383b116756090f7db02a137b788"
server_key_iv: "3742a99f8ecdcd2e59149582a6f57eb6"
server_key_version: "encryptionkey@0"
I20260811 17:14:24.902007 29524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:24.926723 29524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:24.930819 29524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:24.931113 29524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:24.939965 29524 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.188:44127
I20260811 17:14:24.940320 29708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.188:44127 every 8 connection(s)
I20260811 17:14:24.941285 29524 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/data/info.pb
I20260811 17:14:24.942495 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 29524
I20260811 17:14:24.945003 29709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:24.949901 29709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:14:24 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.54.188@KRBTEST.COM
Aug 11 17:14:24 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468464, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.188@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM
I20260811 17:14:24.959412 29711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.944175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58768 (local address 127.21.54.188:44127)
0811 17:14:24.944654 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:24.944675 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:24.944902 (+ 227us) server_negotiation.cc:407] Connection header received
0811 17:14:24.945747 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:24.945768 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:24.946117 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:24.946419 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:24.947413 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.948611 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.949811 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.950345 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.950542 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.953673 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:24.953680 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:24.953697 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:24.953728 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:24.955831 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.956409 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.956415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.956420 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.956607 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.956919 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.956923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.956925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.957525 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:24.957743 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:24.958079 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 17:14:24.958393 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":173,"threads_started":1}
I20260811 17:14:24.963078 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.952293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50694 (local address 127.21.54.190:38223)
0811 17:14:24.952448 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:24.952453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:24.952752 (+ 299us) server_negotiation.cc:407] Connection header received
0811 17:14:24.952932 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:24.952935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:24.953016 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:24.953096 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:24.954336 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.954946 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.956220 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.956594 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.956704 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.959284 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:24.959287 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:24.959289 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:24.959316 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:24.961085 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.961628 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.961631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.961632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.961720 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.962050 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.962055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.962057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.962284 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:24.962403 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:24.962763 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 17:14:24.962899 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 17:14:24.963129 29712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.952081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:50694)
0811 17:14:24.952626 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:24.952668 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:24.952824 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:24.953256 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:24.953261 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:24.953682 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:24.953865 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.953877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.955096 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.955099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.956085 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.956094 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.956797 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.956803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.956917 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.957476 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:24.957490 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:24.959105 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:24.961205 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.961212 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.961218 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.961511 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.961850 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.961853 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.961854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.961935 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.962400 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:24.962406 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:24.962535 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 17:14:24.962700 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 17:14:24.962900 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":186,"threads_started":1}
I20260811 17:14:24.964633 29709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:14:24 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468464, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.188@KRBTEST.COM for oryx/127.21.54.189@KRBTEST.COM
I20260811 17:14:24.974663 29712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.965181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:42221 (local address 127.0.0.1:39672)
0811 17:14:24.965328 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:24.965340 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:24.965411 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:24.965943 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:24.965946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:24.966119 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:24.966235 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.966240 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.967265 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.967268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.967832 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.967845 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.968449 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.968454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.968606 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.969147 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:24.969160 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:24.970630 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:24.972798 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.972802 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.972804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.973036 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.973410 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.973413 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.973414 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.973506 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.974040 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:24.974042 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:24.974141 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 17:14:24.974235 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 17:14:24.974362 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 17:14:24.974677 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.965329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39672 (local address 127.21.54.189:42221)
0811 17:14:24.965691 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:24.965695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:24.965709 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:14:24.965751 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:24.965754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:24.965798 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:24.965882 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:24.966393 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.967130 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.967981 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.968243 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.968417 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.970798 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:24.970802 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:24.970804 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:24.970832 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:24.972681 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.973159 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.973161 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.973163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.973289 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.973650 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.973658 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.973661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.973908 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:24.974034 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:24.974348 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 17:14:24.974504 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":143,"threads_started":1}
I20260811 17:14:24.975835 29709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:14:24 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468464, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.188@KRBTEST.COM for oryx/127.21.54.188@KRBTEST.COM
I20260811 17:14:24.985054 29712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.976655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44127 (local address 127.0.0.1:58770)
0811 17:14:24.976849 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:24.976866 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:24.976951 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:24.977227 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:24.977230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:24.977409 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:24.977488 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.977492 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.978659 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.978662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.979184 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:24.979191 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.979633 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.979636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:24.979723 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.980198 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:24.980210 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:24.981741 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:24.983419 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.983422 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.983424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.983658 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.983930 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:24.983932 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:24.983934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:24.984042 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:24.984455 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:24.984458 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:24.984638 (+ 180us) client_negotiation.cc:769] Sending connection context
0811 17:14:24.984739 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 17:14:24.984871 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":4}
I20260811 17:14:24.985062 29711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:24.976894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58770 (local address 127.21.54.188:44127)
0811 17:14:24.977062 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:24.977065 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:24.977074 (+ 9us) server_negotiation.cc:407] Connection header received
0811 17:14:24.977099 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:24.977101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:24.977132 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:24.977217 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:24.977600 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.978522 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.979301 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:24.979526 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:24.979634 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:24.981880 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:24.981883 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:24.981885 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:24.981908 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:24.983295 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.983758 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.983760 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.983761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.983838 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:24.984151 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:24.984157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:24.984160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:24.984358 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:24.984439 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:24.984800 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 17:14:24.984933 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":66}
I20260811 17:14:24.988647 29709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: Bootstrap starting.
I20260811 17:14:24.991533 29709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:24.992440 29709 log.cc:824] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:24.994575 29709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: No bootstrap required, opened a new log
I20260811 17:14:24.997792 29709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } }
I20260811 17:14:24.998114 29709 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:24.998159 29709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0613d8d679e648219d0996a3849a3668, State: Initialized, Role: FOLLOWER
I20260811 17:14:24.998569 29709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } }
I20260811 17:14:24.999079 29715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } } }
I20260811 17:14:24.999214 29715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:14:24.999922 29709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 17:14:25.005434 29728 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:14:25.005553 29728 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:14:25.009541 29524 master_runner.cc:428] Detected that this master 0613d8d679e648219d0996a3849a3668 is joining an existing cluster
I20260811 17:14:25.009626 29524 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.54.188:44127
I20260811 17:14:25.013370 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 28924
I20260811 17:14:25.026161 29715 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:14:25.026387 29715 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } }
I20260811 17:14:25.027655 29715 leader_election.cc:288] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8aa7804a9c154946b9a8aad1210f7cf7 (127.21.54.190:38223), e72959b06b32490fb14d04885db83c44 (127.21.54.189:42221)
I20260811 17:14:25.028126 28861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0613d8d679e648219d0996a3849a3668" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" is_pre_election: true
I20260811 17:14:25.028368 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0613d8d679e648219d0996a3849a3668" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72959b06b32490fb14d04885db83c44" is_pre_election: true
I20260811 17:14:25.028615 29577 raft_consensus.cc:1799] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 1 LEARNER]: Handling vote request from an unknown peer 0613d8d679e648219d0996a3849a3668
I20260811 17:14:25.028766 29577 raft_consensus.cc:2396] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 0613d8d679e648219d0996a3849a3668 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260811 17:14:25.029163 29653 leader_election.cc:302] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0613d8d679e648219d0996a3849a3668; no voters: 8aa7804a9c154946b9a8aad1210f7cf7, e72959b06b32490fb14d04885db83c44
I20260811 17:14:25.030351 29715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:25.031735 29715 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 17:14:25.035109 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:37479
--local_ip_for_outbound_sockets=127.21.54.129
--tserver_master_addrs=127.21.54.190:38223
--webserver_port=37823
--webserver_interface=127.21.54.129
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.129.keytab
--principal=oryx/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:46121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.54.190:38223,127.21.54.189:42221
--tserver_master_addrs=127.21.54.190:38223,127.21.54.189:42221,127.21.54.188:44127 with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:14:25.050693 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.013067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39686 (local address 127.21.54.189:42221)
0811 17:14:25.013204 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.013207 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.026562 (+ 13355us) server_negotiation.cc:407] Connection header received
0811 17:14:25.027423 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.027427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.027496 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.027553 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.028671 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.029508 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.030765 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.031175 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.031284 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.045806 (+ 14522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:25.045812 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:25.045818 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:25.045864 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:25.047836 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.049308 (+ 1472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.049313 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.049315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.049406 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.049717 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.049720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.049722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.049940 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:25.050031 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.050372 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.050516 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 17:14:25.051193 29737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.012175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44127 (local address 127.0.0.1:58784)
0811 17:14:25.012892 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.012910 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.013064 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.013372 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.013375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.022402 (+ 9027us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:25.022941 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.022959 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.024170 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.024175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.025141 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.025156 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.025961 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.025964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.026067 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.026665 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:25.026679 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:25.027153 (+ 474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:25.047455 (+ 20302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.047458 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.047460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.047980 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.048267 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.048269 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.048271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.048345 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.048813 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:25.048816 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:25.050709 (+ 1893us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.050837 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.051001 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":132,"threads_started":1}
I20260811 17:14:25.051749 29711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.012610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58784 (local address 127.21.54.188:44127)
0811 17:14:25.012740 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.012745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.012967 (+ 222us) server_negotiation.cc:407] Connection header received
0811 17:14:25.013192 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.013196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.013246 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.013337 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.023127 (+ 9790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.024020 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.025278 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.025671 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.025762 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.027290 (+ 1528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:25.027297 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:25.027303 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:25.027334 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:25.047312 (+ 19978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.048093 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.048097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.048099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.048181 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.048424 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.048427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.048429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.048680 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:25.048773 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.051427 (+ 2654us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.051561 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 17:14:25.051936 29738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.021979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:42221 (local address 127.0.0.1:39686)
0811 17:14:25.026460 (+ 4481us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.026476 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.027289 (+ 813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.027712 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.027715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.027934 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:25.028528 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.028537 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.029651 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.029657 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.030615 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.030629 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.031328 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.031335 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.031432 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.031962 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:25.031982 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:25.045463 (+ 13481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:25.048910 (+ 3447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.048913 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.048915 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.049159 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.049513 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.049516 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.049518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.049616 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.050091 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:25.050094 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:25.050190 (+ 96us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.051608 (+ 1418us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.051729 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4267,"thread_start_us":230,"threads_started":1}
I20260811 17:14:25.054248 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.012381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50710 (local address 127.21.54.190:38223)
0811 17:14:25.012570 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.012574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.022996 (+ 10422us) server_negotiation.cc:407] Connection header received
0811 17:14:25.023203 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.023206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.023262 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.023317 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.024022 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.024860 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.026092 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.026453 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.026556 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.048502 (+ 21946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:25.048542 (+ 40us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:25.048545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:25.048580 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:25.050410 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.052897 (+ 2487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.052901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.052903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.053018 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.053320 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.053323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.053325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.053538 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:25.053621 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.053947 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.054092 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 17:14:25.054788 29736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.012616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:50710)
0811 17:14:25.022860 (+ 10244us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.022879 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.023084 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.023461 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.023464 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.023731 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:25.023863 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.023871 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.025023 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.025026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.025943 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.025952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.026896 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.026899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.027001 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.027450 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:25.027464 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:25.048289 (+ 20825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:25.051208 (+ 2919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.051211 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.051214 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.052756 (+ 1542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.053136 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.053139 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.053141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.053222 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.053676 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:25.053679 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:25.053767 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.054458 (+ 691us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.054612 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10147,"thread_start_us":90,"threads_started":1}
Aug 11 17:14:25 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468463, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.190@KRBTEST.COM for oryx/127.21.54.188@KRBTEST.COM
I20260811 17:14:25.069541 29513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.059042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44127 (local address 127.0.0.1:58792)
0811 17:14:25.059223 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.059237 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.059320 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.059765 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.059769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.059988 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:25.060110 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.060117 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.061222 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.061226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.062103 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.062116 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.062815 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.062819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.062921 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.063468 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:25.063483 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:25.065210 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:25.067423 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.067427 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.067429 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.067709 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.068028 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.068031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.068033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.068123 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.068969 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:25.068972 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:25.069079 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.069180 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.069344 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260811 17:14:25.070051 29711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.059117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58792 (local address 127.21.54.188:44127)
0811 17:14:25.059475 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.059479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.059493 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:14:25.059567 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.059569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.059628 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.059711 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.060241 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.061098 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.062260 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.062569 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.062674 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.065402 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:25.065406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:25.065409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:25.065448 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:25.067280 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.067829 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.067833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.067835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.067921 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.068222 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.068225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.068227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.068816 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:25.068919 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.069781 (+ 862us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.069905 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274}
I20260811 17:14:25.071033 28847 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.54.188:44127: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0613d8d679e648219d0996a3849a3668"
member_type: NON_VOTER
last_known_addr {
host: "127.21.54.188"
port: 44127
}
attrs {
promote: true
}
}
dest_uuid: "8aa7804a9c154946b9a8aad1210f7cf7"
cas_config_opid_index: 5
I20260811 17:14:25.071656 28847 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: NON_VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: true } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: NON_VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: true } }
I20260811 17:14:25.073577 29518 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 0613d8d679e648219d0996a3849a3668 (127.21.54.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: NON_VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: true } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: NON_VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: true } } }
I20260811 17:14:25.075157 29518 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.54.188:44127
I20260811 17:14:25.075389 29518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: NON_VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: true } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: NON_VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: true } } }
I20260811 17:14:25.075495 29518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:25.076231 29524 master.cc:537] Master@127.21.54.188:44127 shutting down...
I20260811 17:14:25.077235 29679 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 1 FOLLOWER]: Refusing update from remote peer 8aa7804a9c154946b9a8aad1210f7cf7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 17:14:25.079094 29519 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: NON_VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 17:14:25.081082 28834 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 -> Peer 0613d8d679e648219d0996a3849a3668 (127.21.54.188:44127): Couldn't send request to peer 0613d8d679e648219d0996a3849a3668. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 17:14:25.082954 29524 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:25.083331 29524 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:25.083495 29524 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: stopping tablet replica
I20260811 17:14:25.083914 29513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.073205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:42221 (local address 127.0.0.1:39688)
0811 17:14:25.073347 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.073362 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.073446 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.077210 (+ 3764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.077213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.077235 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:25.077331 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.077338 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.078493 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.078498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.082210 (+ 3712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.082227 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.083443 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.083447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.083544 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.083602 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.083681 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.083772 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 17:14:25.084378 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.073462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39688 (local address 127.21.54.189:42221)
0811 17:14:25.076782 (+ 3320us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.076787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.076804 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:14:25.076862 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.076865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.076934 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.077054 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:25.077469 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.078322 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.082384 (+ 4062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.083091 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.083214 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.083371 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.084167 (+ 796us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.084259 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3218}
I20260811 17:14:25.084877 29577 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 1 LEARNER]: Refusing update from remote peer 8aa7804a9c154946b9a8aad1210f7cf7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 17:14:25.085381 29519 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: NON_VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 17:14:25.087379 29577 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 0613d8d679e648219d0996a3849a3668 (127.21.54.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: NON_VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: true } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: NON_VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: true } } }
I20260811 17:14:25.089700 29741 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: NON_VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: true } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: NON_VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: true } } }
I20260811 17:14:25.089879 29741 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [sys.catalog]: This master's current role is: LEARNER
I20260811 17:14:25.089996 29619 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8aa7804a9c154946b9a8aad1210f7cf7. Latest consensus state: current_term: 1 leader_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: NON_VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: true } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: NON_VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: true } } }
I20260811 17:14:25.090098 29619 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [sys.catalog]: This master's current role is: LEARNER
I20260811 17:14:25.090301 29524 master.cc:552] Master@127.21.54.188:44127 shutdown complete.
I20260811 17:14:25.090382 29524 master_runner.cc:305] Clearing existing system tablet
I20260811 17:14:25.090689 29524 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:25.090799 29524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:25.091348 29519 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7: attempting to promote NON_VOTER e72959b06b32490fb14d04885db83c44 to VOTER
I20260811 17:14:25.091899 29519 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: NON_VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: true } }
I20260811 17:14:25.092897 29524 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 17:14:25.093703 29239 heartbeater.cc:503] Master 127.21.54.190:38223 was elected leader, sending a full tablet report...
I20260811 17:14:25.093957 29743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:25.093971 29577 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 1 LEARNER]: Refusing update from remote peer 8aa7804a9c154946b9a8aad1210f7cf7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 17:14:25.094266 29524 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:25.094376 29524 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 17:14:25.094409 29524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/wal
uuid: "0613d8d679e648219d0996a3849a3668"
format_stamp: "Formatted at 2026-08-11 17:14:24 on dist-test-slave-fd7l"
server_key: "c9254383b116756090f7db02a137b788"
server_key_iv: "3742a99f8ecdcd2e59149582a6f57eb6"
server_key_version: "encryptionkey@0"
I20260811 17:14:25.094394 29516 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 17:14:25.095685 29513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.093392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44127 (local address 127.0.0.1:58804)
0811 17:14:25.095497 (+ 2105us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.095594 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2010}
I20260811 17:14:25.096444 29513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.096210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44127 (local address 127.0.0.1:58808)
0811 17:14:25.096336 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.096381 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 17:14:25.096594 28834 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 -> Peer 0613d8d679e648219d0996a3849a3668 (127.21.54.188:44127): Couldn't send request to peer 0613d8d679e648219d0996a3849a3668. Status: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44127: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 17:14:25.096644 29524 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:25.097591 29516 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e72959b06b32490fb14d04885db83c44 (127.21.54.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: NON_VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: true } } }
I20260811 17:14:25.099159 29516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: NON_VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: true } } }
I20260811 17:14:25.099277 29516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:25.099400 29516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: NON_VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: true } } }
I20260811 17:14:25.099488 29516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:25.099776 29524 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 17:14:25.099912 29524 log.cc:1196] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 17:14:25.100332 29524 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: Deleting consensus metadata
I20260811 17:14:25.100363 29577 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e72959b06b32490fb14d04885db83c44 (127.21.54.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: NON_VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: true } } }
I20260811 17:14:25.100647 29524 master_runner.cc:315] Copying system tablet from 127.21.54.190:38223
I20260811 17:14:25.102236 29524 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: tablet copy: Beginning tablet copy session from remote peer at address 127.21.54.190:38223
I20260811 17:14:25.102407 29619 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: NON_VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: true } } }
I20260811 17:14:25.102550 29619 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:14:25.117311 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.105002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:50720)
0811 17:14:25.108743 (+ 3741us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.108761 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.108905 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.109218 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.109221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.109517 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:25.109654 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.109662 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.110789 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.110793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.111670 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.111682 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.112279 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.112283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.112389 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.113037 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:25.113053 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:25.113506 (+ 453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:25.115584 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.115588 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.115590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.115849 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.116158 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.116161 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.116163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.116259 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.116734 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:25.116737 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:25.116847 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.116960 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.117125 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3642,"thread_start_us":68,"threads_started":1}
I20260811 17:14:25.117327 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.105106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50720 (local address 127.21.54.190:38223)
0811 17:14:25.105221 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.105226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.108875 (+ 3649us) server_negotiation.cc:407] Connection header received
0811 17:14:25.109015 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.109019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.109077 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.109175 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.109785 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.110663 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.111799 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.112146 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.112257 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.113672 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:25.113677 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:25.113680 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:25.113709 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:25.115441 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.115954 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.115958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.115959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.116055 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.116357 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.116360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.116362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.116623 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:25.116725 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.117038 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.117166 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 17:14:25.118183 28864 tablet_copy_service.cc:140] P 8aa7804a9c154946b9a8aad1210f7cf7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0613d8d679e648219d0996a3849a3668 ({username='oryx', principal='oryx/127.21.54.188@KRBTEST.COM'} at 127.0.0.1:50720)
I20260811 17:14:25.118294 28864 tablet_copy_service.cc:161] P 8aa7804a9c154946b9a8aad1210f7cf7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0613d8d679e648219d0996a3849a3668 at {username='oryx', principal='oryx/127.21.54.188@KRBTEST.COM'} at 127.0.0.1:50720: session id = 0613d8d679e648219d0996a3849a3668-00000000000000000000000000000000
I20260811 17:14:25.119454 28864 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 17:14:25.120937 29524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 17:14:25.123567 29524 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: tablet copy: Starting download of 0 data blocks...
I20260811 17:14:25.123759 29524 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: tablet copy: Starting download of 1 WAL segments...
I20260811 17:14:25.125630 29524 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 17:14:25.127707 28864 tablet_copy_service.cc:342] P 8aa7804a9c154946b9a8aad1210f7cf7: Request end of tablet copy session 0613d8d679e648219d0996a3849a3668-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.54.188@KRBTEST.COM'} at 127.0.0.1:50720
I20260811 17:14:25.127804 28864 tablet_copy_service.cc:433] P 8aa7804a9c154946b9a8aad1210f7cf7: ending tablet copy session 0613d8d679e648219d0996a3849a3668-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0613d8d679e648219d0996a3849a3668
I20260811 17:14:25.137079 29524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:25.138918 29751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:25.138937 29753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:25.139474 29758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:25.139632 29524 server_base.cc:1034] running on GCE node
I20260811 17:14:25.139889 29524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:25.140307 29524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:25.141459 29524 hybrid_clock.cc:648] HybridClock initialized: now 1786468465141446 us; error 32 us; skew 500 ppm
Aug 11 17:14:25 dist-test-slave-fd7l krb5kdc[28797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468465, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:25.143447 29524 init.cc:377] Logged in from keytab as oryx/127.21.54.188@KRBTEST.COM (short username oryx)
I20260811 17:14:25.144011 29524 webserver.cc:460] Webserver started at http://127.21.54.188:41157/ using document root <none> and password file <none>
I20260811 17:14:25.144261 29524 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:25.144318 29524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:25.146003 29524 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 17:14:25.147167 29765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:25.147481 29524 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 17:14:25.147574 29524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/wal
uuid: "0613d8d679e648219d0996a3849a3668"
format_stamp: "Formatted at 2026-08-11 17:14:24 on dist-test-slave-fd7l"
server_key: "c9254383b116756090f7db02a137b788"
server_key_iv: "3742a99f8ecdcd2e59149582a6f57eb6"
server_key_version: "encryptionkey@0"
I20260811 17:14:25.147670 29524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:25.160288 29524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:25.163060 29524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:25.168967 29524 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.188:44127
I20260811 17:14:25.169003 29832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.188:44127 every 8 connection(s)
I20260811 17:14:25.169678 29524 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/master-2/data/info.pb
I20260811 17:14:25.171680 29834 sys_catalog.cc:263] Verifying existing consensus state
I20260811 17:14:25.172933 29834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: Bootstrap starting.
I20260811 17:14:25.185851 29834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:14:25.186187 29834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: Bootstrap complete.
I20260811 17:14:25.186826 29834 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: NON_VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: true } }
I20260811 17:14:25.186990 29834 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0613d8d679e648219d0996a3849a3668, State: Initialized, Role: LEARNER
I20260811 17:14:25.187114 29834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: NON_VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: true } }
I20260811 17:14:25.187531 29838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: NON_VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: true } } }
I20260811 17:14:25.187758 29838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [sys.catalog]: This master's current role is: LEARNER
I20260811 17:14:25.187641 29834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:25.190871 29853 catalog_manager.cc:1259] Loaded cluster ID: 59c29e5bbd3b4e5ebe4dee805aa789d1
I20260811 17:14:25.190940 29853 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: loading cluster ID for follower catalog manager: success
I20260811 17:14:25.193259 29853 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: acquiring CA information for follower catalog manager: success
W20260811 17:14:25.194372 29739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:25.194665 29739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:25.194754 29739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 17:14:25.194864 29853 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 17:14:25.199338 29739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:25.199426 29739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:25.199532 29739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:14:25.205511 29739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:46121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.129.keytab
--principal=oryx/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:37479
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=37823
--tserver_master_addrs=127.21.54.190:38223,127.21.54.189:42221,127.21.54.188:44127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:25.206743 29739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:25.207857 29739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:25.218194 29859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:25.218357 29857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:25.218182 29856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:25.218645 29739 server_base.cc:1034] running on GCE node
I20260811 17:14:25.219136 29739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:25.219821 29739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:25.221021 29739 hybrid_clock.cc:648] HybridClock initialized: now 1786468465220997 us; error 35 us; skew 500 ppm
Aug 11 17:14:25 dist-test-slave-fd7l krb5kdc[28797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468465, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:25.224448 29739 init.cc:377] Logged in from keytab as oryx/127.21.54.129@KRBTEST.COM (short username oryx)
I20260811 17:14:25.225659 29739 webserver.cc:460] Webserver started at http://127.21.54.129:37823/ using document root <none> and password file <none>
I20260811 17:14:25.226271 29739 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:25.226392 29739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:25.231040 29739 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 17:14:25.233558 29868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:25.235191 29739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:25.235347 29739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "af046522b683455f8989be6e327b0099"
format_stamp: "Formatted at 2026-08-11 17:14:23 on dist-test-slave-fd7l"
server_key: "7c0499ada3b8275dfc99e608fbf203b7"
server_key_iv: "7d9ee891db56f6b3cc153b80e95792ac"
server_key_version: "encryptionkey@0"
I20260811 17:14:25.235970 29739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:25.254733 29739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:25.258438 29739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:25.258679 29739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:25.259343 29739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:25.260284 29739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:25.260360 29739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:25.260432 29739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:25.260490 29739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:25.272619 29739 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:37479
I20260811 17:14:25.272688 30005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:37479 every 8 connection(s)
I20260811 17:14:25.273873 29739 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:14:25.274427 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 29739
I20260811 17:14:25.274641 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 29084
I20260811 17:14:25.286922 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:42439
--local_ip_for_outbound_sockets=127.21.54.130
--tserver_master_addrs=127.21.54.190:38223
--webserver_port=43139
--webserver_interface=127.21.54.130
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.130.keytab
--principal=oryx/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:46121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.54.190:38223,127.21.54.189:42221
--tserver_master_addrs=127.21.54.190:38223,127.21.54.189:42221,127.21.54.188:44127 with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 17:14:25 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468465, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.188@KRBTEST.COM
Aug 11 17:14:25 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468465, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM
Aug 11 17:14:25 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468465, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.189@KRBTEST.COM
I20260811 17:14:25.295284 30013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.277859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:54811 (local address 127.21.54.188:44127)
0811 17:14:25.278497 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.278502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.279784 (+ 1282us) server_negotiation.cc:407] Connection header received
0811 17:14:25.279977 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.279983 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.280056 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.280124 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.281757 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.282961 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.285122 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.285582 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.285691 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.289999 (+ 4308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:25.290004 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:25.290007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:25.290039 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:25.292426 (+ 2387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.293180 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.293184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.293186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.293288 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.293770 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.293773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.293775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.294027 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:25.294105 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.294957 (+ 852us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.295105 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":98,"threads_started":1}
I20260811 17:14:25.295282 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.278557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:59079 (local address 127.21.54.190:38223)
0811 17:14:25.278706 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.278713 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.278732 (+ 19us) server_negotiation.cc:407] Connection header received
0811 17:14:25.279285 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.279289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.279350 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.279423 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.281774 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.282646 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.285114 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.285479 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.285569 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.290435 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:25.290440 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:25.290444 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:25.290479 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:25.292279 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.293181 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.293184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.293186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.293288 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.293770 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.293773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.293775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.294014 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:25.294105 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.294957 (+ 852us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.295104 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 17:14:25.295940 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.278247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:36889 (local address 127.21.54.189:42221)
0811 17:14:25.278377 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.278382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.278772 (+ 390us) server_negotiation.cc:407] Connection header received
0811 17:14:25.279284 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.279289 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.279351 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.279414 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.281773 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.282640 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.285102 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.285472 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.285568 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.291023 (+ 5455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:25.291030 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:25.291036 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:25.291075 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:25.293272 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.293960 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.293967 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.293970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.294087 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.294493 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.294500 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.294503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.294791 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:25.294916 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.295526 (+ 610us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.295683 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:14:25.297324 30014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.278158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:42221 (local address 127.21.54.129:36889)
0811 17:14:25.278674 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.278689 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.279012 (+ 323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.279913 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.279922 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.281019 (+ 1097us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:25.281569 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.281590 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.282862 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.282866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.284939 (+ 2073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.284945 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.285817 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.285822 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.286209 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.287165 (+ 956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:25.287193 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:25.290807 (+ 3614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:25.293445 (+ 2638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.293450 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.293454 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.293792 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.294238 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.294243 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.294246 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.294357 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.294994 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:25.295000 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:25.295154 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.296201 (+ 1047us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.296403 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":616,"thread_start_us":83,"threads_started":1}
I20260811 17:14:25.297323 30011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.276868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.129:59079)
0811 17:14:25.277711 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.277791 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.279012 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.279913 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.279922 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.280680 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:25.281569 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.281590 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.282862 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.282866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.284930 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.284944 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.285817 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.285822 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.286234 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.287605 (+ 1371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:25.287624 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:25.290242 (+ 2618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:25.292434 (+ 2192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.292441 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.292463 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.293052 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.293410 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.293413 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.293415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.293651 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.294181 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:25.294189 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:25.294643 (+ 454us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.295561 (+ 918us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.296021 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":741,"spinlock_wait_cycles":54272,"thread_start_us":229,"threads_started":1}
I20260811 17:14:25.297319 30015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.276868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44127 (local address 127.21.54.129:54811)
0811 17:14:25.279679 (+ 2811us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.279697 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.279862 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.280303 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.280308 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.280825 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:25.281571 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.281593 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.283132 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.283138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.284964 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.284979 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.285893 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.285896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.286258 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.287327 (+ 1069us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:25.287347 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:25.289787 (+ 2440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:25.292567 (+ 2780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.292572 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.292578 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.293052 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.293410 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.293413 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.293415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.293651 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.294181 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:25.294189 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:25.294643 (+ 454us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.295812 (+ 1169us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.296274 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2663,"mutex_wait_us":2704,"spinlock_wait_cycles":25600,"thread_start_us":141,"threads_started":1}
I20260811 17:14:25.299235 30009 heartbeater.cc:347] Connected to a master server at 127.21.54.189:42221
I20260811 17:14:25.299346 30007 heartbeater.cc:347] Connected to a master server at 127.21.54.190:38223
I20260811 17:14:25.299245 30006 heartbeater.cc:347] Connected to a master server at 127.21.54.188:44127
I20260811 17:14:25.299562 30006 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:25.299562 30009 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:25.299561 30007 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:25.300380 30007 heartbeater.cc:511] Master 127.21.54.190:38223 requested a full tablet report, sending...
I20260811 17:14:25.300354 30009 heartbeater.cc:511] Master 127.21.54.189:42221 requested a full tablet report, sending...
I20260811 17:14:25.300372 30006 heartbeater.cc:511] Master 127.21.54.188:44127 requested a full tablet report, sending...
I20260811 17:14:25.302022 28847 ts_manager.cc:194] Re-registered known tserver with Master: af046522b683455f8989be6e327b0099 (127.21.54.129:37479)
I20260811 17:14:25.302177 29568 ts_manager.cc:194] Registered new tserver with Master: af046522b683455f8989be6e327b0099 (127.21.54.129:37479)
I20260811 17:14:25.302263 29787 ts_manager.cc:194] Registered new tserver with Master: af046522b683455f8989be6e327b0099 (127.21.54.129:37479)
I20260811 17:14:25.302858 28847 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:59079
I20260811 17:14:25.328020 29399 heartbeater.cc:503] Master 127.21.54.190:38223 was elected leader, sending a full tablet report...
W20260811 17:14:25.418493 30016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:25.418886 30016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:25.418996 30016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:25.423305 30016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:25.423427 30016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:25.423537 30016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:14:25.429817 30016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:46121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.130.keytab
--principal=oryx/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:42439
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=43139
--tserver_master_addrs=127.21.54.190:38223,127.21.54.189:42221,127.21.54.188:44127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:25.431286 30016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:25.433184 30016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:25.446135 30025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:25.446110 30022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:25.446084 30016 server_base.cc:1034] running on GCE node
W20260811 17:14:25.446079 30023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:25.447077 30016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:25.447956 30016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:25.449156 30016 hybrid_clock.cc:648] HybridClock initialized: now 1786468465449109 us; error 84 us; skew 500 ppm
Aug 11 17:14:25 dist-test-slave-fd7l krb5kdc[28797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468465, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:25.452783 30016 init.cc:377] Logged in from keytab as oryx/127.21.54.130@KRBTEST.COM (short username oryx)
I20260811 17:14:25.453789 30016 webserver.cc:460] Webserver started at http://127.21.54.130:43139/ using document root <none> and password file <none>
I20260811 17:14:25.454407 30016 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:25.454496 30016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:25.459278 30016 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:14:25.461889 30034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:25.463457 30016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:25.463629 30016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "6127ec870149479e80c2b8d80e1e20a1"
format_stamp: "Formatted at 2026-08-11 17:14:24 on dist-test-slave-fd7l"
server_key: "515e621984c45b53eff8f2f2ef90ec6b"
server_key_iv: "ca9aa4fb70c10d39f9144f83678cea3a"
server_key_version: "encryptionkey@0"
I20260811 17:14:25.464500 30016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:25.487102 30016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:25.490914 30016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:25.491146 30016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:25.491842 30016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:25.492877 30016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:25.492946 30016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:25.493017 30016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:25.493058 30016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:25.504422 30016 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:42439
I20260811 17:14:25.504448 30171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:42439 every 8 connection(s)
I20260811 17:14:25.505632 30016 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:14:25.506767 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 30016
I20260811 17:14:25.506959 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 29246
I20260811 17:14:25.521675 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:41509
--local_ip_for_outbound_sockets=127.21.54.131
--tserver_master_addrs=127.21.54.190:38223
--webserver_port=38541
--webserver_interface=127.21.54.131
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.131.keytab
--principal=oryx/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:46121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.54.190:38223,127.21.54.189:42221
--tserver_master_addrs=127.21.54.190:38223,127.21.54.189:42221,127.21.54.188:44127 with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 17:14:25 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468465, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM
Aug 11 17:14:25 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468465, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.189@KRBTEST.COM
Aug 11 17:14:25 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468465, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.188@KRBTEST.COM
I20260811 17:14:25.531427 30013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.514348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:38473 (local address 127.21.54.188:44127)
0811 17:14:25.514493 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.514500 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.515163 (+ 663us) server_negotiation.cc:407] Connection header received
0811 17:14:25.516102 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.516109 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.516184 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.516299 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.518221 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.519089 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.523260 (+ 4171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.523633 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.523736 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.526663 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:25.526668 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:25.526672 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:25.526707 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:25.528776 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.529346 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.529349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.529352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.529617 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.529912 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.529915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.529916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.530140 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:25.530233 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.531099 (+ 866us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.531248 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:14:25.531525 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.512955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:42417 (local address 127.21.54.190:38223)
0811 17:14:25.513093 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.513100 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.517471 (+ 4371us) server_negotiation.cc:407] Connection header received
0811 17:14:25.517650 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.517654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.517705 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.517759 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.518385 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.519154 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.521434 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.521763 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.521858 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.526563 (+ 4705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:25.526567 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:25.526572 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:25.526610 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:25.529045 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.529636 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.529642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.529646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.529755 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.530129 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.530134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.530137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.530416 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:25.530516 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.531134 (+ 618us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.531293 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 17:14:25.532668 30177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.512378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.130:42417)
0811 17:14:25.517382 (+ 5004us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.517396 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.517538 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.517910 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.517913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.518121 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:25.518249 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.518256 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.519291 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.519294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.521287 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.521298 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.522101 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.522107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.522338 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.523077 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:25.523101 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:25.526386 (+ 3285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:25.529204 (+ 2818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.529209 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.529214 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.529500 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.529884 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.529889 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.529892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.529991 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.530586 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:25.530592 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:25.530775 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.531691 (+ 916us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.531868 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4935,"mutex_wait_us":365,"spinlock_wait_cycles":22400,"thread_start_us":268,"threads_started":1}
I20260811 17:14:25.532666 30176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.508769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:42221 (local address 127.21.54.130:59289)
0811 17:14:25.517065 (+ 8296us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.517079 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.517186 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.517584 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.517590 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.517831 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:25.518044 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.518058 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.519292 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.519298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.521283 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.521293 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.522100 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.522104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.522318 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.522963 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:25.522989 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:25.525881 (+ 2892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:25.528712 (+ 2831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.528724 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.528747 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.529224 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.529574 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.529577 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.529579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.529780 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.530293 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:25.530301 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:25.530770 (+ 469us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.531017 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.531384 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8219,"mutex_wait_us":307,"spinlock_wait_cycles":6144,"thread_start_us":347,"threads_started":1}
I20260811 17:14:25.532666 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.508237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44127 (local address 127.21.54.130:38473)
0811 17:14:25.514966 (+ 6729us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.515028 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.515860 (+ 832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.516581 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.516593 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.517199 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:25.518027 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.518049 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.519291 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.519294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.523096 (+ 3802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.523107 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.523938 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.523944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.524042 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.524626 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:25.524642 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:25.526369 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:25.528910 (+ 2541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.528913 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.528915 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.529225 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.529725 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.529728 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.529730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.529808 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.530293 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:25.530301 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:25.530770 (+ 469us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.531596 (+ 826us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.531734 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6480,"mutex_wait_us":2264,"spinlock_wait_cycles":23296,"thread_start_us":794,"threads_started":1}
I20260811 17:14:25.533072 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.516742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:59289 (local address 127.21.54.189:42221)
0811 17:14:25.516914 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.516920 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.517162 (+ 242us) server_negotiation.cc:407] Connection header received
0811 17:14:25.517274 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.517280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.517339 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.517424 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.518226 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.519112 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.521426 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.521671 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.521792 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.526184 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:25.526188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:25.526191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:25.526228 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:25.528550 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.529346 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.529349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.529352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.529453 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.529894 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.529899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.529900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.530149 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:25.530233 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.532792 (+ 2559us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.532925 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 17:14:25.534498 30174 heartbeater.cc:347] Connected to a master server at 127.21.54.189:42221
I20260811 17:14:25.534472 30172 heartbeater.cc:347] Connected to a master server at 127.21.54.188:44127
I20260811 17:14:25.534741 30174 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:25.534744 30172 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:25.534436 30173 heartbeater.cc:347] Connected to a master server at 127.21.54.190:38223
I20260811 17:14:25.535311 30173 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:25.535434 30172 heartbeater.cc:511] Master 127.21.54.188:44127 requested a full tablet report, sending...
I20260811 17:14:25.535481 30174 heartbeater.cc:511] Master 127.21.54.189:42221 requested a full tablet report, sending...
I20260811 17:14:25.535465 30173 heartbeater.cc:511] Master 127.21.54.190:38223 requested a full tablet report, sending...
I20260811 17:14:25.536742 28847 ts_manager.cc:194] Re-registered known tserver with Master: 6127ec870149479e80c2b8d80e1e20a1 (127.21.54.130:42439)
I20260811 17:14:25.536814 29568 ts_manager.cc:194] Registered new tserver with Master: 6127ec870149479e80c2b8d80e1e20a1 (127.21.54.130:42439)
I20260811 17:14:25.536906 29787 ts_manager.cc:194] Registered new tserver with Master: 6127ec870149479e80c2b8d80e1e20a1 (127.21.54.130:42439)
I20260811 17:14:25.537539 28847 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:42417
I20260811 17:14:25.558001 29513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.553914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44127 (local address 127.0.0.1:58810)
0811 17:14:25.554091 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.554109 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.554222 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.554605 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.554608 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.554628 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:25.554760 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.554767 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.555524 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.555528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.556758 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.556773 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.557549 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.557553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.557647 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.557676 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.557764 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.557848 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 17:14:25.557999 30013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.554006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58810 (local address 127.21.54.188:44127)
0811 17:14:25.554163 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.554167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.554184 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:14:25.554387 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.554390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.554458 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.554576 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:25.554853 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.555399 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.556900 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.557361 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.557517 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.557646 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.557812 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.557887 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 17:14:25.558759 29791 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 17:14:25.559264 29838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8aa7804a9c154946b9a8aad1210f7cf7. Latest consensus state: current_term: 1 leader_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: NON_VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: true } } }
I20260811 17:14:25.559501 29838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [sys.catalog]: This master's current role is: LEARNER
W20260811 17:14:25.651609 30179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:25.651878 30179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:25.651921 30179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:25.655997 30179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:25.656066 30179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:25.656167 30179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:14:25.661741 30179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:46121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.131.keytab
--principal=oryx/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:41509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=38541
--tserver_master_addrs=127.21.54.190:38223,127.21.54.189:42221,127.21.54.188:44127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:25.662901 30179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:25.664008 30179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:25.674440 30188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:25.674433 30185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:25.674431 30190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:25.674489 30179 server_base.cc:1034] running on GCE node
I20260811 17:14:25.675172 30179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:25.675827 30179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:25.677000 30179 hybrid_clock.cc:648] HybridClock initialized: now 1786468465676957 us; error 52 us; skew 500 ppm
Aug 11 17:14:25 dist-test-slave-fd7l krb5kdc[28797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468465, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:25.680617 30179 init.cc:377] Logged in from keytab as oryx/127.21.54.131@KRBTEST.COM (short username oryx)
I20260811 17:14:25.681654 30179 webserver.cc:460] Webserver started at http://127.21.54.131:38541/ using document root <none> and password file <none>
I20260811 17:14:25.682327 30179 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:25.682441 30179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:25.687011 30179 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:14:25.689647 30198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:25.691185 30179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:25.691340 30179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "23f3969ead3b4e70b1014825b3a82790"
format_stamp: "Formatted at 2026-08-11 17:14:24 on dist-test-slave-fd7l"
server_key: "e1ea614cff37f921a9fea76b923c2f5a"
server_key_iv: "166d8d2afdb31c6d74c9c2ff4a9d9132"
server_key_version: "encryptionkey@0"
I20260811 17:14:25.691933 30179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:25.725998 30179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:25.729924 30179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:25.730268 30179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:25.731029 30179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:25.732020 30179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:25.732092 30179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:25.732182 30179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:25.732213 30179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:25.743391 30179 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:41509
I20260811 17:14:25.743469 30335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:41509 every 8 connection(s)
I20260811 17:14:25.744477 30179 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:14:25.752847 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 30179
Aug 11 17:14:25 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468465, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.189@KRBTEST.COM
Aug 11 17:14:25 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468465, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.188@KRBTEST.COM
Aug 11 17:14:25 dist-test-slave-fd7l krb5kdc[28797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468465, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM
I20260811 17:14:25.762028 30013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.746505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:52025 (local address 127.21.54.188:44127)
0811 17:14:25.746660 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.746665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.747646 (+ 981us) server_negotiation.cc:407] Connection header received
0811 17:14:25.748921 (+ 1275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.748925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.748979 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.749034 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.750472 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.751017 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.752722 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.753070 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.753157 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.757391 (+ 4234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:25.757395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:25.757398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:25.757428 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:25.759503 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.760413 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.760422 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.760426 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.760553 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.760930 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.760935 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.760937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.761209 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:25.761308 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.761706 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.761838 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 17:14:25.762789 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.746610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:56541 (local address 127.21.54.189:42221)
0811 17:14:25.746778 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.746784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.748335 (+ 1551us) server_negotiation.cc:407] Connection header received
0811 17:14:25.748907 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.748911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.748969 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.749039 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.750522 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.751100 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.752675 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.753060 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.753144 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.756692 (+ 3548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:25.756697 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:25.756700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:25.756736 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:25.758648 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.759440 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.759444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.759446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.759710 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.760219 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.760222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.760224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.760543 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:25.760640 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.762474 (+ 1834us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.762633 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 17:14:25.764096 30343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.746953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:42221 (local address 127.21.54.131:56541)
0811 17:14:25.748217 (+ 1264us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.748241 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.748673 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.749235 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.749243 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.749800 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:25.750338 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.750349 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.751227 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.751233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.752468 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.752479 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.753363 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.753366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.753551 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.754298 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:25.754317 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:25.756421 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:25.758866 (+ 2445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.758874 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.758894 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.759301 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.759837 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.759841 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.759843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.760096 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.760701 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:25.760708 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:25.761067 (+ 359us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.761288 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.761710 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1177,"mutex_wait_us":217,"spinlock_wait_cycles":18176,"thread_start_us":161,"threads_started":1}
I20260811 17:14:25.764223 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.748094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:57047 (local address 127.21.54.190:38223)
0811 17:14:25.748234 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.748238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.748739 (+ 501us) server_negotiation.cc:407] Connection header received
0811 17:14:25.749173 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.749177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.749228 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.749307 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.751209 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.751992 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.754934 (+ 2942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.755259 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.755397 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.759978 (+ 4581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:25.759984 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:25.759987 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:25.760014 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:25.761838 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.762972 (+ 1134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.762976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.762978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.763082 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.763366 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.763370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.763371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.763580 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:25.763685 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.763943 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.764065 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 17:14:25.764240 30344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.747906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.131:57047)
0811 17:14:25.748638 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.748653 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.748762 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.749338 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.749341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.749800 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:25.750338 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.750349 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.752128 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.752132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.753138 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.753148 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.755406 (+ 2258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.755409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.755499 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.756013 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:25.756028 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:25.759852 (+ 3824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:25.762343 (+ 2491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.762346 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.762348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.762609 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.763170 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.763176 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.763179 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.763273 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.763674 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:25.763677 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:25.763774 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.763878 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.764024 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":121,"spinlock_wait_cycles":62976,"thread_start_us":82,"threads_started":1}
I20260811 17:14:25.764295 30341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.746954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44127 (local address 127.21.54.131:52025)
0811 17:14:25.747456 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.747511 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.748666 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.749235 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.749244 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.749760 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:25.750322 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.750337 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.751185 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.751188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.752465 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.752475 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.753461 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.753467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.753609 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.754413 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:25.754429 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:25.756946 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:25.759997 (+ 3051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.760000 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.760001 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.760288 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.760696 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:25.760700 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:25.760704 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:25.760801 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:25.761377 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:25.761382 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:25.761477 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.762700 (+ 1223us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.762867 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":170,"thread_start_us":126,"threads_started":1}
I20260811 17:14:25.766183 30336 heartbeater.cc:347] Connected to a master server at 127.21.54.188:44127
I20260811 17:14:25.766218 30337 heartbeater.cc:347] Connected to a master server at 127.21.54.190:38223
I20260811 17:14:25.766458 30338 heartbeater.cc:347] Connected to a master server at 127.21.54.189:42221
I20260811 17:14:25.766518 30336 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:25.766520 30337 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:25.766520 30338 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:25.767268 30338 heartbeater.cc:511] Master 127.21.54.189:42221 requested a full tablet report, sending...
I20260811 17:14:25.767951 30336 heartbeater.cc:511] Master 127.21.54.188:44127 requested a full tablet report, sending...
I20260811 17:14:25.768638 30337 heartbeater.cc:511] Master 127.21.54.190:38223 requested a full tablet report, sending...
I20260811 17:14:25.768853 29787 ts_manager.cc:194] Registered new tserver with Master: 23f3969ead3b4e70b1014825b3a82790 (127.21.54.131:41509)
I20260811 17:14:25.768877 29568 ts_manager.cc:194] Registered new tserver with Master: 23f3969ead3b4e70b1014825b3a82790 (127.21.54.131:41509)
I20260811 17:14:25.769331 28847 ts_manager.cc:194] Re-registered known tserver with Master: 23f3969ead3b4e70b1014825b3a82790 (127.21.54.131:41509)
I20260811 17:14:25.770056 28847 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:57047
I20260811 17:14:25.772500 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.756902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39704 (local address 127.21.54.189:42221)
0811 17:14:25.757218 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.757223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.757797 (+ 574us) server_negotiation.cc:407] Connection header received
0811 17:14:25.758267 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.758273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.758334 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.758431 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.759495 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.760393 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.761692 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.762051 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.762201 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.764185 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:25.764191 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:25.764198 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:25.764232 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:25.769971 (+ 5739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.770505 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.770510 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.770512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.770620 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.771048 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.771052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.771054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.771366 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:25.771507 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.772164 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.772304 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":132,"threads_started":1}
I20260811 17:14:25.776782 30013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.757394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58822 (local address 127.21.54.188:44127)
0811 17:14:25.762229 (+ 4835us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.762233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.763187 (+ 954us) server_negotiation.cc:407] Connection header received
0811 17:14:25.764564 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.764578 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.764640 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.764726 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.767131 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.767888 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.770012 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.770314 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.770397 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.772332 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:25.772336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:25.772339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:25.772369 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:25.774457 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.775038 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.775041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.775043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.775133 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.775543 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.775547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.775549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.775823 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:25.776054 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.776372 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.776602 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4774,"thread_start_us":102,"threads_started":1}
I20260811 17:14:25.776880 30351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.756647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50736 (local address 127.21.54.190:38223)
0811 17:14:25.757134 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.757140 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.757901 (+ 761us) server_negotiation.cc:407] Connection header received
0811 17:14:25.757953 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.757957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.758012 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.758117 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.759504 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.762347 (+ 2843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.768010 (+ 5663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.768396 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.768525 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.772491 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:25.772496 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:25.772500 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:25.772551 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:25.774699 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.775252 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.775258 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.775259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.775370 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:25.775768 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:25.775772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:25.775774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:25.775988 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:25.776150 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.776425 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.776622 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":102,"threads_started":1}
I20260811 17:14:25.779948 28847 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:14:25.783495 28847 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:14:25.786785 29519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: NON_VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: true } } }
I20260811 17:14:25.786983 29519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:25.791891 30359 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7: attempting to promote NON_VOTER 0613d8d679e648219d0996a3849a3668 to VOTER
I20260811 17:14:25.792276 30359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } }
I20260811 17:14:25.793573 29577 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 1 FOLLOWER]: Refusing update from remote peer 8aa7804a9c154946b9a8aad1210f7cf7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 17:14:25.794011 29519 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 17:14:25.794571 29791 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 1 LEARNER]: Refusing update from remote peer 8aa7804a9c154946b9a8aad1210f7cf7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 17:14:25.795622 29519 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 17:14:25.795953 30359 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 0613d8d679e648219d0996a3849a3668 (127.21.54.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } } }
I20260811 17:14:25.796574 29577 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 0613d8d679e648219d0996a3849a3668 (127.21.54.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } } }
I20260811 17:14:25.798663 30358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } } }
I20260811 17:14:25.798791 30358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:25.798714 29519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } } }
I20260811 17:14:25.798866 29519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:25.798543 30359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } } }
I20260811 17:14:25.799005 30359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:25.798740 29791 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 0613d8d679e648219d0996a3849a3668 (127.21.54.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } } }
I20260811 17:14:25.800868 30368 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } } }
I20260811 17:14:25.801023 30368 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:14:25.801544 29838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } } }
I20260811 17:14:25.801674 29838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:14:25.808943 30371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.802218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59586 (local address 127.21.54.130:42439)
0811 17:14:25.802564 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.802579 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.803368 (+ 789us) server_negotiation.cc:407] Connection header received
0811 17:14:25.803445 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.803449 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.803651 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.803834 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:25.804085 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.805320 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.807500 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.808262 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.808392 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.808589 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.808708 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.808811 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":100,"threads_started":1}
I20260811 17:14:25.808934 29513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.801363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:41509 (local address 127.0.0.1:48608)
0811 17:14:25.801657 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.801681 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.801775 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.804764 (+ 2989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.804767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.804789 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:25.804872 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.804878 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.806238 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.806242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.807404 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.807411 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.808548 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.808552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.808640 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.808663 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.808721 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.808795 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":111}
I20260811 17:14:25.808871 30370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.802015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:42439 (local address 127.0.0.1:59586)
0811 17:14:25.802927 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.802942 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.803060 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.803797 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.803800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.803821 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:25.803943 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.803952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.805464 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.805468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.807364 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.807374 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.808394 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.808398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.808484 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.808539 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.808610 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.808700 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":847,"mutex_wait_us":672,"thread_start_us":139,"threads_started":1}
I20260811 17:14:25.809415 30369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.801514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48608 (local address 127.21.54.131:41509)
0811 17:14:25.804236 (+ 2722us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.804250 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.804273 (+ 23us) server_negotiation.cc:407] Connection header received
0811 17:14:25.804348 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.804352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.804563 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.804758 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:25.805001 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.806110 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.807540 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.808369 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.808946 (+ 577us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.809122 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.809175 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.809265 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2520,"thread_start_us":159,"threads_started":1}
I20260811 17:14:25.809805 30372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.802812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:37479 (local address 127.0.0.1:33542)
0811 17:14:25.803081 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.803095 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.803188 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.804075 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.804081 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.804100 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:25.804194 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.804200 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.805441 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.805450 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.807379 (+ 1929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.807395 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.808753 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.808757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.808833 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.808854 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.808929 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.809611 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":26,"thread_start_us":130,"threads_started":1}
I20260811 17:14:25.810392 30375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.803060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33542 (local address 127.21.54.129:37479)
0811 17:14:25.803323 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.803330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.803357 (+ 27us) server_negotiation.cc:407] Connection header received
0811 17:14:25.803447 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.803456 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.803689 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.803906 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:25.804325 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.805322 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.807549 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.808369 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.808498 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.808691 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.810143 (+ 1452us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.810265 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":92,"threads_started":1}
I20260811 17:14:25.812219 30256 tablet_service.cc:1467] Processing CreateTablet for tablet e51497d469cf4269a0a9dd2e62cddeba (DEFAULT_TABLE table=test-table [id=90452d5cd38e4e7c97a1b9a7c5ffa498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:25.813285 30256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51497d469cf4269a0a9dd2e62cddeba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:25.812879 29925 tablet_service.cc:1467] Processing CreateTablet for tablet e51497d469cf4269a0a9dd2e62cddeba (DEFAULT_TABLE table=test-table [id=90452d5cd38e4e7c97a1b9a7c5ffa498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:25.813934 29925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51497d469cf4269a0a9dd2e62cddeba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:25.814561 30090 tablet_service.cc:1467] Processing CreateTablet for tablet e51497d469cf4269a0a9dd2e62cddeba (DEFAULT_TABLE table=test-table [id=90452d5cd38e4e7c97a1b9a7c5ffa498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:25.815578 30090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51497d469cf4269a0a9dd2e62cddeba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:25.820839 30376 tablet_bootstrap.cc:492] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790: Bootstrap starting.
I20260811 17:14:25.823405 30376 tablet_bootstrap.cc:654] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:25.824023 30377 tablet_bootstrap.cc:492] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1: Bootstrap starting.
I20260811 17:14:25.824389 30376 log.cc:824] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:25.824666 30380 tablet_bootstrap.cc:492] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099: Bootstrap starting.
I20260811 17:14:25.826987 30380 tablet_bootstrap.cc:654] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:25.827013 30377 tablet_bootstrap.cc:654] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:25.827836 30377 log.cc:824] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:25.827852 30380 log.cc:824] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:25.830149 30377 tablet_bootstrap.cc:492] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1: No bootstrap required, opened a new log
I20260811 17:14:25.830153 30376 tablet_bootstrap.cc:492] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790: No bootstrap required, opened a new log
I20260811 17:14:25.830369 30377 ts_tablet_manager.cc:1397] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:14:25.830369 30376 ts_tablet_manager.cc:1397] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260811 17:14:25.831677 30380 tablet_bootstrap.cc:492] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099: No bootstrap required, opened a new log
I20260811 17:14:25.831951 30380 ts_tablet_manager.cc:1397] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:14:25.835160 30377 raft_consensus.cc:348] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } } peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } }
I20260811 17:14:25.835491 30377 raft_consensus.cc:374] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:25.835564 30377 raft_consensus.cc:729] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6127ec870149479e80c2b8d80e1e20a1, State: Initialized, Role: FOLLOWER
I20260811 17:14:25.836176 30377 consensus_queue.cc:260] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } } peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } }
I20260811 17:14:25.836576 30380 raft_consensus.cc:348] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } } peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } }
I20260811 17:14:25.836597 30376 raft_consensus.cc:348] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } } peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } }
I20260811 17:14:25.836843 30376 raft_consensus.cc:374] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:25.836900 30376 raft_consensus.cc:729] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23f3969ead3b4e70b1014825b3a82790, State: Initialized, Role: FOLLOWER
I20260811 17:14:25.837000 30380 raft_consensus.cc:374] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:25.837175 30380 raft_consensus.cc:729] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af046522b683455f8989be6e327b0099, State: Initialized, Role: FOLLOWER
I20260811 17:14:25.837463 30377 ts_tablet_manager.cc:1428] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1: Time spent starting tablet: real 0.007s user 0.006s sys 0.002s
I20260811 17:14:25.837345 30376 consensus_queue.cc:260] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } } peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } }
I20260811 17:14:25.837749 30380 consensus_queue.cc:260] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } } peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } }
I20260811 17:14:25.838339 30376 ts_tablet_manager.cc:1428] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260811 17:14:25.839092 30173 heartbeater.cc:503] Master 127.21.54.190:38223 was elected leader, sending a full tablet report...
I20260811 17:14:25.839535 30337 heartbeater.cc:503] Master 127.21.54.190:38223 was elected leader, sending a full tablet report...
I20260811 17:14:25.842204 30380 ts_tablet_manager.cc:1428] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260811 17:14:25.842684 30007 heartbeater.cc:503] Master 127.21.54.190:38223 was elected leader, sending a full tablet report...
I20260811 17:14:25.901770 30386 raft_consensus.cc:482] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:14:25.902050 30386 raft_consensus.cc:504] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } } peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } }
I20260811 17:14:25.903285 30386 leader_election.cc:288] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6127ec870149479e80c2b8d80e1e20a1 (127.21.54.130:42439), 23f3969ead3b4e70b1014825b3a82790 (127.21.54.131:41509)
I20260811 17:14:25.907415 30371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.903995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:36195 (local address 127.21.54.130:42439)
0811 17:14:25.904173 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.904178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.904194 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:14:25.904244 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.904247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.904293 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.904382 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:25.904728 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.905259 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.906398 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.906797 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.906930 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.906988 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.907211 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.907279 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 17:14:25.907478 30389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.903601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:41509 (local address 127.21.54.129:46293)
0811 17:14:25.904037 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.904053 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.904203 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.904488 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.904491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.904531 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:25.904662 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.904667 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.905436 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.905440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.906269 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.906279 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.906928 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.906931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.907071 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.907138 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.907225 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.907301 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":61,"spinlock_wait_cycles":14592,"thread_start_us":173,"threads_started":1}
I20260811 17:14:25.907465 30369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.903755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:46293 (local address 127.21.54.131:41509)
0811 17:14:25.903910 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.903915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.904165 (+ 250us) server_negotiation.cc:407] Connection header received
0811 17:14:25.904326 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.904329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.904382 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.904476 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:25.904806 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.905317 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.906406 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.906800 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.906934 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.906990 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.907257 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.907336 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 17:14:25.907678 30387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.903582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:42439 (local address 127.21.54.129:36195)
0811 17:14:25.903951 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:25.903966 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:25.904105 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:25.904394 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:25.904397 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:25.904413 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:25.904603 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.904610 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.905375 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.905381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.906260 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:25.906270 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.906924 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.906927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:25.907028 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.907101 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 17:14:25.907192 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 17:14:25.907270 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":45,"thread_start_us":154,"threads_started":1}
I20260811 17:14:25.908149 30280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51497d469cf4269a0a9dd2e62cddeba" candidate_uuid: "af046522b683455f8989be6e327b0099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23f3969ead3b4e70b1014825b3a82790" is_pre_election: true
I20260811 17:14:25.908324 30117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51497d469cf4269a0a9dd2e62cddeba" candidate_uuid: "af046522b683455f8989be6e327b0099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6127ec870149479e80c2b8d80e1e20a1" is_pre_election: true
I20260811 17:14:25.908527 30280 raft_consensus.cc:2454] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af046522b683455f8989be6e327b0099 in term 0.
I20260811 17:14:25.908746 30117 raft_consensus.cc:2454] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af046522b683455f8989be6e327b0099 in term 0.
I20260811 17:14:25.909080 29869 leader_election.cc:302] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23f3969ead3b4e70b1014825b3a82790, af046522b683455f8989be6e327b0099; no voters:
I20260811 17:14:25.909406 30386 raft_consensus.cc:2781] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:14:25.909495 30386 raft_consensus.cc:482] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:14:25.909529 30386 raft_consensus.cc:3037] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:25.910667 30386 raft_consensus.cc:504] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } } peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } }
I20260811 17:14:25.911100 30386 leader_election.cc:288] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [CANDIDATE]: Term 1 election: Requested vote from peers 6127ec870149479e80c2b8d80e1e20a1 (127.21.54.130:42439), 23f3969ead3b4e70b1014825b3a82790 (127.21.54.131:41509)
I20260811 17:14:25.911556 30280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51497d469cf4269a0a9dd2e62cddeba" candidate_uuid: "af046522b683455f8989be6e327b0099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23f3969ead3b4e70b1014825b3a82790"
I20260811 17:14:25.911536 30117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51497d469cf4269a0a9dd2e62cddeba" candidate_uuid: "af046522b683455f8989be6e327b0099" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6127ec870149479e80c2b8d80e1e20a1"
I20260811 17:14:25.911674 30280 raft_consensus.cc:3037] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:25.911722 30117 raft_consensus.cc:3037] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:25.913013 30280 raft_consensus.cc:2454] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af046522b683455f8989be6e327b0099 in term 1.
I20260811 17:14:25.913116 30117 raft_consensus.cc:2454] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af046522b683455f8989be6e327b0099 in term 1.
I20260811 17:14:25.913389 29869 leader_election.cc:302] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23f3969ead3b4e70b1014825b3a82790, af046522b683455f8989be6e327b0099; no voters:
I20260811 17:14:25.913617 30386 raft_consensus.cc:2781] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:25.913816 30386 raft_consensus.cc:686] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [term 1 LEADER]: Becoming Leader. State: Replica: af046522b683455f8989be6e327b0099, State: Running, Role: LEADER
I20260811 17:14:25.914240 30386 consensus_queue.cc:237] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } } peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } }
I20260811 17:14:25.918066 28846 catalog_manager.cc:5495] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 reported cstate change: term changed from 0 to 1, leader changed from <none> to af046522b683455f8989be6e327b0099 (127.21.54.129). New cstate: current_term: 1 leader_uuid: "af046522b683455f8989be6e327b0099" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } health_report { overall_health: HEALTHY } } }
I20260811 17:14:25.935034 30375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.931245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33554 (local address 127.21.54.129:37479)
0811 17:14:25.931409 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.931414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.931430 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:14:25.931562 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.931565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.931630 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.931755 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:14:25.932004 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.932623 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.933492 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.933727 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.933842 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.934190 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:14:25.934637 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:14:25.934776 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:25.934818 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 17:14:25.934880 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 17:14:25.941919 30117 raft_consensus.cc:1261] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1 [term 1 FOLLOWER]: Refusing update from remote peer af046522b683455f8989be6e327b0099: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:25.941920 30280 raft_consensus.cc:1261] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790 [term 1 FOLLOWER]: Refusing update from remote peer af046522b683455f8989be6e327b0099: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:25.942775 30390 consensus_queue.cc:1035] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:25.943043 30386 consensus_queue.cc:1035] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:25.952832 30394 mvcc.cc:205] Tried to move back new op lower bound from 7317374836489564160 to 7317374836387897344. Current Snapshot: MvccSnapshot[applied={T|T < 7317374836489564160}]
I20260811 17:14:25.955624 30396 mvcc.cc:205] Tried to move back new op lower bound from 7317374836489564160 to 7317374836387897344. Current Snapshot: MvccSnapshot[applied={T|T < 7317374836489564160}]
I20260811 17:14:25.964359 28845 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:25.964623 28845 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:25.971097 28845 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8aa7804a9c154946b9a8aad1210f7cf7: Sending DeleteTablet for 3 replicas of tablet e51497d469cf4269a0a9dd2e62cddeba
I20260811 17:14:25.972036 30256 tablet_service.cc:1514] Processing DeleteTablet for tablet e51497d469cf4269a0a9dd2e62cddeba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:25 UTC) from {username='oryx', principal='oryx/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:48608
I20260811 17:14:25.971982 30090 tablet_service.cc:1514] Processing DeleteTablet for tablet e51497d469cf4269a0a9dd2e62cddeba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:25 UTC) from {username='oryx', principal='oryx/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:59586
I20260811 17:14:25.972219 29925 tablet_service.cc:1514] Processing DeleteTablet for tablet e51497d469cf4269a0a9dd2e62cddeba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:25 UTC) from {username='oryx', principal='oryx/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:33542
I20260811 17:14:25.972586 30404 tablet_replica.cc:330] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790: stopping tablet replica
I20260811 17:14:25.973050 30404 raft_consensus.cc:2229] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:25.973476 30404 raft_consensus.cc:2258] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:25.973948 30403 tablet_replica.cc:330] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1: stopping tablet replica
I20260811 17:14:25.974308 30403 raft_consensus.cc:2229] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:25.974639 30403 raft_consensus.cc:2258] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:25.974812 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 28812
I20260811 17:14:25.979789 30404 ts_tablet_manager.cc:1905] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:25.983224 30404 ts_tablet_manager.cc:1918] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:14:25.983347 30404 log.cc:1196] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/e51497d469cf4269a0a9dd2e62cddeba
I20260811 17:14:25.983711 30404 ts_tablet_manager.cc:1939] T e51497d469cf4269a0a9dd2e62cddeba P 23f3969ead3b4e70b1014825b3a82790: Deleting consensus metadata
I20260811 17:14:25.983249 30405 tablet_replica.cc:330] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099: stopping tablet replica
I20260811 17:14:25.986613 30403 ts_tablet_manager.cc:1905] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:25.986872 30405 raft_consensus.cc:2229] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:14:25.987484 30405 raft_consensus.cc:2258] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:25.989559 30403 ts_tablet_manager.cc:1918] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:14:25.989681 30403 log.cc:1196] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/e51497d469cf4269a0a9dd2e62cddeba
W20260811 17:14:25.989907 30200 connection.cc:535] server connection from 127.0.0.1:48608 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 17:14:25.990187 30403 ts_tablet_manager.cc:1939] T e51497d469cf4269a0a9dd2e62cddeba P 6127ec870149479e80c2b8d80e1e20a1: Deleting consensus metadata
W20260811 17:14:25.991478 30037 connection.cc:441] server connection from 127.0.0.1:59586 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59586 (request call id 1) could send its response
I20260811 17:14:25.992204 30405 ts_tablet_manager.cc:1905] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:25.995427 30405 ts_tablet_manager.cc:1918] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:14:25.995551 30405 log.cc:1196] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-0/wal/wals/e51497d469cf4269a0a9dd2e62cddeba
I20260811 17:14:25.995961 30405 ts_tablet_manager.cc:1939] T e51497d469cf4269a0a9dd2e62cddeba P af046522b683455f8989be6e327b0099: Deleting consensus metadata
I20260811 17:14:26.003371 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.974202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39714 (local address 127.21.54.189:42221)
0811 17:14:25.974350 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.974356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.981165 (+ 6809us) server_negotiation.cc:407] Connection header received
0811 17:14:25.981463 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.981470 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.981543 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.981611 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.983567 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.984631 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.987950 (+ 3319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.988472 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.988627 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:25.998067 (+ 9440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:25.998071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:25.998074 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:25.998106 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:26.000429 (+ 2323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:26.001052 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:26.001057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:26.001059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:26.001160 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:26.001537 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:26.001542 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:26.001544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:26.002359 (+ 815us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:26.002520 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:26.002963 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 17:14:26.003120 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 17:14:26.004925 29872 connection.cc:441] server connection from 127.0.0.1:33542 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33542 (request call id 1) could send its response
I20260811 17:14:26.005335 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.994490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58846 (local address 127.21.54.188:44127)
0811 17:14:25.994616 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.994620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.995272 (+ 652us) server_negotiation.cc:407] Connection header received
0811 17:14:25.995472 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.995476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.995531 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.995592 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.997221 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.998064 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.999291 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.999633 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:25.999709 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:26.001500 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:26.001504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:26.001507 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:26.001537 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:26.003441 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:26.003928 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:26.003931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:26.003933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:26.004031 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:26.004328 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:26.004331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:26.004333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:26.004567 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:26.004685 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:26.005046 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 17:14:26.005175 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 17:14:26.008764 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.995434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39716 (local address 127.21.54.189:42221)
0811 17:14:25.995576 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.995581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:25.996941 (+ 1360us) server_negotiation.cc:407] Connection header received
0811 17:14:25.997162 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:25.997167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:25.997233 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:25.997307 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:25.998309 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:25.999244 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:26.000717 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:26.001130 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:26.001221 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:26.003059 (+ 1838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:26.003065 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:26.003072 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:26.003101 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:26.005272 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:26.007182 (+ 1910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:26.007186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:26.007187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:26.007278 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:26.007745 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:26.007750 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:26.007753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:26.007966 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:26.008209 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:26.008468 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 17:14:26.008610 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:14:26.014770 30013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:25.974346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58838 (local address 127.21.54.188:44127)
0811 17:14:25.974474 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:25.974480 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:26.005515 (+ 31035us) server_negotiation.cc:407] Connection header received
0811 17:14:26.005596 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:26.005599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:26.005651 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:26.005708 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:26.006608 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:26.007461 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:26.008622 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:26.009024 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:26.009106 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:26.010782 (+ 1676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:26.010786 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:26.010789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:26.010815 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:26.012317 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:26.012921 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:26.012924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:26.012926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:26.013036 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:26.013610 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:26.013615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:26.013617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:26.013876 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:26.014039 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:26.014434 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 17:14:26.014596 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:14:26.960847 30414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:26.960148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.129:38119)
0811 17:14:26.960586 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:26.960710 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":106,"threads_started":1}
W20260811 17:14:26.961249 30007 heartbeater.cc:644] Failed to heartbeat to 127.21.54.190:38223 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
I20260811 17:14:26.963922 30417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:26.963440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.130:46293)
0811 17:14:26.963730 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:26.963813 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":89,"threads_started":1}
I20260811 17:14:26.963946 30415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:26.963440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.131:54441)
0811 17:14:26.963783 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:26.963861 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":92,"threads_started":1}
W20260811 17:14:26.964319 30173 heartbeater.cc:644] Failed to heartbeat to 127.21.54.190:38223 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
W20260811 17:14:26.964465 30337 heartbeater.cc:644] Failed to heartbeat to 127.21.54.190:38223 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
I20260811 17:14:27.471102 30418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8aa7804a9c154946b9a8aad1210f7cf7)
I20260811 17:14:27.471305 30418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } }
I20260811 17:14:27.471390 30420 raft_consensus.cc:482] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8aa7804a9c154946b9a8aad1210f7cf7)
I20260811 17:14:27.471638 30420 raft_consensus.cc:504] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } }
I20260811 17:14:27.472100 30418 leader_election.cc:288] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8aa7804a9c154946b9a8aad1210f7cf7 (127.21.54.190:38223), e72959b06b32490fb14d04885db83c44 (127.21.54.189:42221)
I20260811 17:14:27.472301 30420 leader_election.cc:288] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8aa7804a9c154946b9a8aad1210f7cf7 (127.21.54.190:38223), 0613d8d679e648219d0996a3849a3668 (127.21.54.188:44127)
I20260811 17:14:27.473230 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:27.472315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:51248)
0811 17:14:27.472986 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:27.473100 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":88,"threads_started":1}
I20260811 17:14:27.473831 30425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:27.472618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:51250)
0811 17:14:27.473644 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:27.473727 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":933,"thread_start_us":69,"threads_started":1}
W20260811 17:14:27.473891 29769 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
W20260811 17:14:27.474109 29554 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
I20260811 17:14:27.475296 30425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:27.475036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:51258)
0811 17:14:27.475182 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:27.475229 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260811 17:14:27.475502 29554 leader_election.cc:334] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8aa7804a9c154946b9a8aad1210f7cf7 (127.21.54.190:38223): Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
I20260811 17:14:27.475626 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:27.475166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:51264)
0811 17:14:27.475335 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:27.475435 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260811 17:14:27.475811 29769 leader_election.cc:334] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8aa7804a9c154946b9a8aad1210f7cf7 (127.21.54.190:38223): Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
I20260811 17:14:27.478341 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:27.472704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44127 (local address 127.0.0.1:58852)
0811 17:14:27.473681 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:27.473699 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:27.473812 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:27.474245 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:27.474248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:27.474266 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:27.474383 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:27.474393 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:27.475467 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:27.475470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:27.476762 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:27.476774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:27.477876 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:27.477880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:27.477988 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:27.478026 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 17:14:27.478112 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 17:14:27.478191 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":910,"mutex_wait_us":4,"thread_start_us":103,"threads_started":1}
I20260811 17:14:27.478586 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:27.472740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:42221 (local address 127.0.0.1:39728)
0811 17:14:27.472986 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:27.473006 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:27.473135 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:27.474187 (+ 1052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:27.474191 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:27.474216 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:27.474336 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:27.474347 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:27.475536 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:27.475539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:27.476822 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:27.476852 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:27.478169 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:27.478173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:27.478268 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:27.478291 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 17:14:27.478343 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 17:14:27.478433 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":92,"threads_started":1}
I20260811 17:14:27.478631 30428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:27.472978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58852 (local address 127.21.54.188:44127)
0811 17:14:27.473351 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:27.473358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:27.473914 (+ 556us) server_negotiation.cc:407] Connection header received
0811 17:14:27.473970 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:27.473973 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:27.474020 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:27.474099 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:27.474532 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:27.475326 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:27.476895 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:27.477703 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:27.477895 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:27.477986 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:27.478141 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 17:14:27.478228 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":169,"threads_started":1}
I20260811 17:14:27.479043 30429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:27.473337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39728 (local address 127.21.54.189:42221)
0811 17:14:27.473873 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:27.473879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:27.473897 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:14:27.473968 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:27.473971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:27.474026 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:27.474471 (+ 445us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:27.474589 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:27.475383 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:27.477353 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:27.478025 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:27.478771 (+ 746us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:27.478829 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:27.478866 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 17:14:27.478918 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":463,"thread_start_us":83,"threads_started":1}
I20260811 17:14:27.479387 29791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e72959b06b32490fb14d04885db83c44" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0613d8d679e648219d0996a3849a3668" is_pre_election: true
I20260811 17:14:27.479456 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0613d8d679e648219d0996a3849a3668" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e72959b06b32490fb14d04885db83c44" is_pre_election: true
I20260811 17:14:27.479581 29577 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0613d8d679e648219d0996a3849a3668 in term 1.
I20260811 17:14:27.479650 29791 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e72959b06b32490fb14d04885db83c44 in term 1.
I20260811 17:14:27.480046 29553 leader_election.cc:302] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0613d8d679e648219d0996a3849a3668, e72959b06b32490fb14d04885db83c44; no voters: 8aa7804a9c154946b9a8aad1210f7cf7
I20260811 17:14:27.480283 29769 leader_election.cc:302] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0613d8d679e648219d0996a3849a3668, e72959b06b32490fb14d04885db83c44; no voters: 8aa7804a9c154946b9a8aad1210f7cf7
I20260811 17:14:27.480425 30420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 17:14:27.480487 30420 raft_consensus.cc:482] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8aa7804a9c154946b9a8aad1210f7cf7)
I20260811 17:14:27.480566 30420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 1 FOLLOWER]: Advancing to term 2
I20260811 17:14:27.480572 30418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 17:14:27.480626 30418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8aa7804a9c154946b9a8aad1210f7cf7)
I20260811 17:14:27.480676 30418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 1 FOLLOWER]: Advancing to term 2
I20260811 17:14:27.481766 30418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } }
I20260811 17:14:27.481772 30420 raft_consensus.cc:504] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } }
I20260811 17:14:27.482197 30418 leader_election.cc:288] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [CANDIDATE]: Term 2 election: Requested vote from peers 8aa7804a9c154946b9a8aad1210f7cf7 (127.21.54.190:38223), e72959b06b32490fb14d04885db83c44 (127.21.54.189:42221)
I20260811 17:14:27.482259 30420 leader_election.cc:288] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [CANDIDATE]: Term 2 election: Requested vote from peers 8aa7804a9c154946b9a8aad1210f7cf7 (127.21.54.190:38223), 0613d8d679e648219d0996a3849a3668 (127.21.54.188:44127)
I20260811 17:14:27.482641 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:27.482273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:51282)
0811 17:14:27.482514 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:27.482572 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
I20260811 17:14:27.482621 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0613d8d679e648219d0996a3849a3668" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e72959b06b32490fb14d04885db83c44"
I20260811 17:14:27.482774 29577 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0613d8d679e648219d0996a3849a3668 in current term 2: Already voted for candidate e72959b06b32490fb14d04885db83c44 in this term.
I20260811 17:14:27.483016 29791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e72959b06b32490fb14d04885db83c44" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0613d8d679e648219d0996a3849a3668"
I20260811 17:14:27.483181 29791 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e72959b06b32490fb14d04885db83c44 in current term 2: Already voted for candidate 0613d8d679e648219d0996a3849a3668 in this term.
I20260811 17:14:27.483312 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:27.482713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:51296)
0811 17:14:27.483056 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:27.483187 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262}
I20260811 17:14:27.483927 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:27.483643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:51306)
0811 17:14:27.483805 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:27.483857 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260811 17:14:27.484054 29769 leader_election.cc:334] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8aa7804a9c154946b9a8aad1210f7cf7 (127.21.54.190:38223): Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
I20260811 17:14:27.484078 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:27.483813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:51314)
0811 17:14:27.483964 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:27.484015 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260811 17:14:27.484157 29554 leader_election.cc:334] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8aa7804a9c154946b9a8aad1210f7cf7 (127.21.54.190:38223): Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
I20260811 17:14:27.484177 29769 leader_election.cc:302] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0613d8d679e648219d0996a3849a3668; no voters: 8aa7804a9c154946b9a8aad1210f7cf7, e72959b06b32490fb14d04885db83c44
I20260811 17:14:27.484231 29554 leader_election.cc:302] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e72959b06b32490fb14d04885db83c44; no voters: 0613d8d679e648219d0996a3849a3668, 8aa7804a9c154946b9a8aad1210f7cf7
I20260811 17:14:27.484376 30420 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 17:14:27.484436 30418 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 17:14:27.962719 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:27.962021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.129:45869)
0811 17:14:27.962390 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:27.962596 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":118,"threads_started":1}
I20260811 17:14:27.965564 30434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:27.965082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.130:34599)
0811 17:14:27.965413 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:27.965466 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":97,"threads_started":1}
I20260811 17:14:27.965583 30435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:27.965108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.131:42195)
0811 17:14:27.965388 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:27.965497 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":72,"threads_started":1}
I20260811 17:14:28.964031 30436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:28.963566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.129:39685)
0811 17:14:28.963804 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:28.963902 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":76,"threads_started":1}
I20260811 17:14:28.966889 30437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:28.966335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.130:52699)
0811 17:14:28.966747 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:28.966803 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":77,"threads_started":1}
I20260811 17:14:28.966892 30438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:28.966477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.131:51717)
0811 17:14:28.966735 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:28.966802 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":66,"threads_started":1}
I20260811 17:14:29.061466 30439 raft_consensus.cc:482] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:14:29.061640 30439 raft_consensus.cc:504] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } }
I20260811 17:14:29.062315 30439 leader_election.cc:288] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8aa7804a9c154946b9a8aad1210f7cf7 (127.21.54.190:38223), 0613d8d679e648219d0996a3849a3668 (127.21.54.188:44127)
I20260811 17:14:29.062917 29791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e72959b06b32490fb14d04885db83c44" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0613d8d679e648219d0996a3849a3668" is_pre_election: true
I20260811 17:14:29.063107 29791 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e72959b06b32490fb14d04885db83c44 in term 2.
I20260811 17:14:29.063223 30442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.062670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:51504)
0811 17:14:29.063001 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.063071 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":142,"threads_started":1}
I20260811 17:14:29.063522 29553 leader_election.cc:302] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0613d8d679e648219d0996a3849a3668, e72959b06b32490fb14d04885db83c44; no voters:
I20260811 17:14:29.063720 30439 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 17:14:29.063783 30439 raft_consensus.cc:482] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:14:29.063831 30439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 2 FOLLOWER]: Advancing to term 3
I20260811 17:14:29.064365 30442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.064064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:51508)
0811 17:14:29.064199 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.064274 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 17:14:29.064538 29554 leader_election.cc:334] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8aa7804a9c154946b9a8aad1210f7cf7 (127.21.54.190:38223): Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
I20260811 17:14:29.065218 30439 raft_consensus.cc:504] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } }
I20260811 17:14:29.065583 30439 leader_election.cc:288] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [CANDIDATE]: Term 3 election: Requested vote from peers 8aa7804a9c154946b9a8aad1210f7cf7 (127.21.54.190:38223), 0613d8d679e648219d0996a3849a3668 (127.21.54.188:44127)
I20260811 17:14:29.065891 30442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.065691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:51516)
0811 17:14:29.065782 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.065831 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260811 17:14:29.066026 29791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e72959b06b32490fb14d04885db83c44" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0613d8d679e648219d0996a3849a3668"
I20260811 17:14:29.066195 29791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 2 FOLLOWER]: Advancing to term 3
I20260811 17:14:29.066563 30442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.066358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:51532)
0811 17:14:29.066460 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.066506 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 17:14:29.066737 29554 leader_election.cc:334] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 8aa7804a9c154946b9a8aad1210f7cf7 (127.21.54.190:38223): Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
I20260811 17:14:29.067376 29791 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e72959b06b32490fb14d04885db83c44 in term 3.
I20260811 17:14:29.067746 29553 leader_election.cc:302] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0613d8d679e648219d0996a3849a3668, e72959b06b32490fb14d04885db83c44; no voters: 8aa7804a9c154946b9a8aad1210f7cf7
I20260811 17:14:29.067950 30439 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 17:14:29.068220 30439 raft_consensus.cc:686] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [term 3 LEADER]: Becoming Leader. State: Replica: e72959b06b32490fb14d04885db83c44, State: Running, Role: LEADER
I20260811 17:14:29.068542 30439 consensus_queue.cc:237] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } }
I20260811 17:14:29.069841 30444 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e72959b06b32490fb14d04885db83c44. Latest consensus state: current_term: 3 leader_uuid: "e72959b06b32490fb14d04885db83c44" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } } }
I20260811 17:14:29.070004 30444 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:29.070356 30446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:14:29.073105 30446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:14:29.073542 30446 catalog_manager.cc:1259] Loaded cluster ID: 59c29e5bbd3b4e5ebe4dee805aa789d1
I20260811 17:14:29.073588 30446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:14:29.074204 30446 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:14:29.074611 30446 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: Loaded TSK: 0
I20260811 17:14:29.075330 30446 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:14:29.090538 29567 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39716:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:14:29.093997 29567 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:14:29.097507 30442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.097236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:51540)
0811 17:14:29.097366 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.097429 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 17:14:29.097661 29791 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [term 3 FOLLOWER]: Refusing update from remote peer e72959b06b32490fb14d04885db83c44: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 17:14:29.098228 30444 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 17:14:29.098366 30442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.098081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.0.0.1:51548)
0811 17:14:29.098243 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.098302 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260811 17:14:29.098505 29554 consensus_peers.cc:487] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 -> Peer 8aa7804a9c154946b9a8aad1210f7cf7 (127.21.54.190:38223): Couldn't send request to peer 8aa7804a9c154946b9a8aad1210f7cf7. Status: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 17:14:29.099946 30451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e72959b06b32490fb14d04885db83c44. Latest consensus state: current_term: 3 leader_uuid: "e72959b06b32490fb14d04885db83c44" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } } }
I20260811 17:14:29.100101 30451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:14:29.101475 30444 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "e72959b06b32490fb14d04885db83c44" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } } }
I20260811 17:14:29.101640 30444 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:29.102301 30451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "e72959b06b32490fb14d04885db83c44" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8aa7804a9c154946b9a8aad1210f7cf7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38223 } } peers { permanent_uuid: "e72959b06b32490fb14d04885db83c44" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 42221 } attrs { promote: false } } peers { permanent_uuid: "0613d8d679e648219d0996a3849a3668" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44127 } attrs { promote: false } } }
I20260811 17:14:29.102439 30451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0613d8d679e648219d0996a3849a3668 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:14:29.112799 30459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.108367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48614 (local address 127.21.54.131:41509)
0811 17:14:29.108665 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:29.108670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:29.108683 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:14:29.108737 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:29.108739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:29.108780 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:29.108873 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:29.109178 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.110050 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.111499 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.112132 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.112247 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:29.112315 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:29.112563 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 17:14:29.112649 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":85,"threads_started":1}
I20260811 17:14:29.112783 30442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.108232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:41509 (local address 127.0.0.1:48614)
0811 17:14:29.108397 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.108414 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:29.108551 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:29.108895 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:29.108897 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:29.108915 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:29.109058 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:29.109063 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.110146 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.110150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:29.111349 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:29.111359 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.112270 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.112274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:29.112373 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:29.112403 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 17:14:29.112492 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 17:14:29.112602 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 17:14:29.113523 30256 tablet_service.cc:1467] Processing CreateTablet for tablet 43953cf9bfa84acea4fe4ab92892e9ed (DEFAULT_TABLE table=test-table [id=73071370a13546b7a1e6c0ad84b94012]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:29.113855 30256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43953cf9bfa84acea4fe4ab92892e9ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:29.114301 30460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.109115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:42439 (local address 127.0.0.1:59602)
0811 17:14:29.109578 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.109595 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:29.109689 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:29.109992 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:29.109994 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:29.110013 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:29.110098 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:29.110102 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.111441 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.111444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:29.112651 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:29.112661 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.113803 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.113807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:29.113957 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:29.113985 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 17:14:29.114065 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 17:14:29.114150 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":1,"thread_start_us":97,"threads_started":1}
I20260811 17:14:29.114312 30463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.109476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59602 (local address 127.21.54.130:42439)
0811 17:14:29.109769 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:29.109773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:29.109787 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:14:29.109838 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:29.109843 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:29.109887 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:29.109976 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:29.110235 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.111113 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.113045 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.113591 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.113740 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:29.113820 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:29.114099 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 17:14:29.114182 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":116,"threads_started":1}
I20260811 17:14:29.114511 30462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.108747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:37479 (local address 127.0.0.1:33558)
0811 17:14:29.109339 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.109353 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:29.109430 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:29.110610 (+ 1180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:29.110613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:29.110635 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:29.110738 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:29.110744 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.111790 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.111794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:29.112957 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:29.112969 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.114085 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.114089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:29.114187 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:29.114228 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 17:14:29.114284 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 17:14:29.114366 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":1,"thread_start_us":92,"threads_started":1}
I20260811 17:14:29.115051 30461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.109024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33558 (local address 127.21.54.129:37479)
0811 17:14:29.110289 (+ 1265us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:29.110294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:29.110307 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:14:29.110356 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:29.110359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:29.110405 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:29.110522 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:29.110885 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.111647 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.113346 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.113939 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.114704 (+ 765us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:29.114769 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:29.114808 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 17:14:29.114881 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1185,"thread_start_us":95,"threads_started":1}
I20260811 17:14:29.115571 29925 tablet_service.cc:1467] Processing CreateTablet for tablet 43953cf9bfa84acea4fe4ab92892e9ed (DEFAULT_TABLE table=test-table [id=73071370a13546b7a1e6c0ad84b94012]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:29.115861 29925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43953cf9bfa84acea4fe4ab92892e9ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:29.116838 30464 tablet_bootstrap.cc:492] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790: Bootstrap starting.
I20260811 17:14:29.117683 30464 tablet_bootstrap.cc:654] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:29.118381 30466 tablet_bootstrap.cc:492] T 43953cf9bfa84acea4fe4ab92892e9ed P af046522b683455f8989be6e327b0099: Bootstrap starting.
I20260811 17:14:29.118624 30090 tablet_service.cc:1467] Processing CreateTablet for tablet 43953cf9bfa84acea4fe4ab92892e9ed (DEFAULT_TABLE table=test-table [id=73071370a13546b7a1e6c0ad84b94012]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:29.118921 30090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43953cf9bfa84acea4fe4ab92892e9ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:29.119256 30464 tablet_bootstrap.cc:492] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790: No bootstrap required, opened a new log
I20260811 17:14:29.119345 30464 ts_tablet_manager.cc:1397] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:14:29.119397 30466 tablet_bootstrap.cc:654] T 43953cf9bfa84acea4fe4ab92892e9ed P af046522b683455f8989be6e327b0099: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:29.119902 30464 raft_consensus.cc:348] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } } peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } }
I20260811 17:14:29.120065 30464 raft_consensus.cc:374] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:29.120110 30464 raft_consensus.cc:729] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23f3969ead3b4e70b1014825b3a82790, State: Initialized, Role: FOLLOWER
I20260811 17:14:29.120287 30464 consensus_queue.cc:260] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } } peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } }
I20260811 17:14:29.120713 30464 ts_tablet_manager.cc:1428] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:14:29.120890 30466 tablet_bootstrap.cc:492] T 43953cf9bfa84acea4fe4ab92892e9ed P af046522b683455f8989be6e327b0099: No bootstrap required, opened a new log
I20260811 17:14:29.120996 30466 ts_tablet_manager.cc:1397] T 43953cf9bfa84acea4fe4ab92892e9ed P af046522b683455f8989be6e327b0099: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:14:29.121449 30466 raft_consensus.cc:348] T 43953cf9bfa84acea4fe4ab92892e9ed P af046522b683455f8989be6e327b0099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } } peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } }
I20260811 17:14:29.121580 30466 raft_consensus.cc:374] T 43953cf9bfa84acea4fe4ab92892e9ed P af046522b683455f8989be6e327b0099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:29.121619 30466 raft_consensus.cc:729] T 43953cf9bfa84acea4fe4ab92892e9ed P af046522b683455f8989be6e327b0099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af046522b683455f8989be6e327b0099, State: Initialized, Role: FOLLOWER
I20260811 17:14:29.121771 30466 consensus_queue.cc:260] T 43953cf9bfa84acea4fe4ab92892e9ed P af046522b683455f8989be6e327b0099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } } peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } }
I20260811 17:14:29.121850 30470 tablet_bootstrap.cc:492] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1: Bootstrap starting.
I20260811 17:14:29.122188 30466 ts_tablet_manager.cc:1428] T 43953cf9bfa84acea4fe4ab92892e9ed P af046522b683455f8989be6e327b0099: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:14:29.122896 30436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.122605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.129:58539)
0811 17:14:29.122760 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.122822 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 17:14:29.123162 30438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.122507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.131:57899)
0811 17:14:29.123034 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.123088 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440}
I20260811 17:14:29.124446 30470 tablet_bootstrap.cc:654] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:29.125442 30436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.125223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.129:33173)
0811 17:14:29.125321 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.125372 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 17:14:29.126000 30009 heartbeater.cc:503] Master 127.21.54.189:42221 was elected leader, sending a full tablet report...
I20260811 17:14:29.126678 30470 tablet_bootstrap.cc:492] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1: No bootstrap required, opened a new log
I20260811 17:14:29.126796 30470 ts_tablet_manager.cc:1397] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 17:14:29.127264 30470 raft_consensus.cc:348] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } } peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } }
I20260811 17:14:29.127398 30470 raft_consensus.cc:374] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:29.127440 30470 raft_consensus.cc:729] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6127ec870149479e80c2b8d80e1e20a1, State: Initialized, Role: FOLLOWER
I20260811 17:14:29.127583 30470 consensus_queue.cc:260] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } } peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } }
I20260811 17:14:29.127934 30470 ts_tablet_manager.cc:1428] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 17:14:29.128598 30437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.128331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.130:36849)
0811 17:14:29.128428 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.128497 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 17:14:29.129668 30174 heartbeater.cc:503] Master 127.21.54.189:42221 was elected leader, sending a full tablet report...
I20260811 17:14:29.129801 30437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.129416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.130:36683)
0811 17:14:29.129652 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.129708 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260811 17:14:29.164920 30473 raft_consensus.cc:482] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:14:29.165138 30473 raft_consensus.cc:504] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } } peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } }
I20260811 17:14:29.166290 30473 leader_election.cc:288] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23f3969ead3b4e70b1014825b3a82790 (127.21.54.131:41509), af046522b683455f8989be6e327b0099 (127.21.54.129:37479)
I20260811 17:14:29.171739 30461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.166693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:59925 (local address 127.21.54.129:37479)
0811 17:14:29.166835 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:29.166840 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:29.167155 (+ 315us) server_negotiation.cc:407] Connection header received
0811 17:14:29.167362 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:29.167367 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:29.167420 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:29.167576 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:29.167753 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.168472 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.170453 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.171104 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.171237 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:29.171290 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:29.171500 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 17:14:29.171602 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 17:14:29.171782 30474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.166598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:37479 (local address 127.21.54.130:59925)
0811 17:14:29.167051 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.167063 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:29.167186 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:29.167526 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:29.167529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:29.167547 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:29.167648 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:29.167653 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.168677 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.168680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:29.170323 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:29.170336 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.171259 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.171262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:29.171341 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:29.171364 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 17:14:29.171539 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 17:14:29.171619 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":190,"thread_start_us":166,"threads_started":1}
I20260811 17:14:29.172497 30437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.166557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:41509 (local address 127.21.54.130:34147)
0811 17:14:29.166764 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.166780 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:29.166896 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:29.167272 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:29.167278 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:29.167299 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:29.167414 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:29.167419 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.168659 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.168666 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:29.170323 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:29.170336 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.172110 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.172113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:29.172225 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:29.172250 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 17:14:29.172336 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 17:14:29.172409 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":131}
I20260811 17:14:29.172601 30459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.166651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:34147 (local address 127.21.54.131:41509)
0811 17:14:29.166813 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:29.166820 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:29.166839 (+ 19us) server_negotiation.cc:407] Connection header received
0811 17:14:29.167106 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:29.167109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:29.167157 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:29.167245 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:29.167612 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.168430 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.171353 (+ 2923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.171976 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.172101 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:29.172148 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:29.172363 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 17:14:29.172438 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 17:14:29.173293 30280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43953cf9bfa84acea4fe4ab92892e9ed" candidate_uuid: "6127ec870149479e80c2b8d80e1e20a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23f3969ead3b4e70b1014825b3a82790" is_pre_election: true
I20260811 17:14:29.173381 29947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43953cf9bfa84acea4fe4ab92892e9ed" candidate_uuid: "6127ec870149479e80c2b8d80e1e20a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af046522b683455f8989be6e327b0099" is_pre_election: true
I20260811 17:14:29.173454 30280 raft_consensus.cc:2454] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6127ec870149479e80c2b8d80e1e20a1 in term 0.
I20260811 17:14:29.173657 29947 raft_consensus.cc:2454] T 43953cf9bfa84acea4fe4ab92892e9ed P af046522b683455f8989be6e327b0099 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6127ec870149479e80c2b8d80e1e20a1 in term 0.
I20260811 17:14:29.173848 30035 leader_election.cc:302] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23f3969ead3b4e70b1014825b3a82790, 6127ec870149479e80c2b8d80e1e20a1; no voters:
I20260811 17:14:29.174322 30473 raft_consensus.cc:2781] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:14:29.174383 30473 raft_consensus.cc:482] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:14:29.174422 30473 raft_consensus.cc:3037] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:29.175761 30473 raft_consensus.cc:504] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } } peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } }
I20260811 17:14:29.176146 30473 leader_election.cc:288] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [CANDIDATE]: Term 1 election: Requested vote from peers 23f3969ead3b4e70b1014825b3a82790 (127.21.54.131:41509), af046522b683455f8989be6e327b0099 (127.21.54.129:37479)
I20260811 17:14:29.176558 29947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43953cf9bfa84acea4fe4ab92892e9ed" candidate_uuid: "6127ec870149479e80c2b8d80e1e20a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af046522b683455f8989be6e327b0099"
I20260811 17:14:29.176582 30280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43953cf9bfa84acea4fe4ab92892e9ed" candidate_uuid: "6127ec870149479e80c2b8d80e1e20a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23f3969ead3b4e70b1014825b3a82790"
I20260811 17:14:29.176704 30280 raft_consensus.cc:3037] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:29.176714 29947 raft_consensus.cc:3037] T 43953cf9bfa84acea4fe4ab92892e9ed P af046522b683455f8989be6e327b0099 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:29.177800 30280 raft_consensus.cc:2454] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6127ec870149479e80c2b8d80e1e20a1 in term 1.
I20260811 17:14:29.178026 29947 raft_consensus.cc:2454] T 43953cf9bfa84acea4fe4ab92892e9ed P af046522b683455f8989be6e327b0099 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6127ec870149479e80c2b8d80e1e20a1 in term 1.
I20260811 17:14:29.178133 30035 leader_election.cc:302] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23f3969ead3b4e70b1014825b3a82790, 6127ec870149479e80c2b8d80e1e20a1; no voters:
I20260811 17:14:29.178309 30473 raft_consensus.cc:2781] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:29.178635 30473 raft_consensus.cc:686] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [term 1 LEADER]: Becoming Leader. State: Replica: 6127ec870149479e80c2b8d80e1e20a1, State: Running, Role: LEADER
I20260811 17:14:29.178929 30473 consensus_queue.cc:237] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } } peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } }
I20260811 17:14:29.179332 30437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.179092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.130:39741)
0811 17:14:29.179198 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.179261 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 17:14:29.181787 29567 catalog_manager.cc:5495] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6127ec870149479e80c2b8d80e1e20a1 (127.21.54.130). New cstate: current_term: 1 leader_uuid: "6127ec870149479e80c2b8d80e1e20a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6127ec870149479e80c2b8d80e1e20a1" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42439 } health_report { overall_health: HEALTHY } } }
I20260811 17:14:29.199404 30463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.195616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59606 (local address 127.21.54.130:42439)
0811 17:14:29.195783 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:29.195787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:29.195799 (+ 12us) server_negotiation.cc:407] Connection header received
0811 17:14:29.195901 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:29.195903 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:29.195963 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:29.196085 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:14:29.196343 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.196947 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.197853 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:29.198092 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:29.198205 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:29.198504 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:14:29.198975 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:14:29.199111 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:29.199178 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 17:14:29.199242 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 17:14:29.202438 30280 raft_consensus.cc:1261] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790 [term 1 FOLLOWER]: Refusing update from remote peer 6127ec870149479e80c2b8d80e1e20a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:29.202728 29947 raft_consensus.cc:1261] T 43953cf9bfa84acea4fe4ab92892e9ed P af046522b683455f8989be6e327b0099 [term 1 FOLLOWER]: Refusing update from remote peer 6127ec870149479e80c2b8d80e1e20a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:29.203122 30473 consensus_queue.cc:1035] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23f3969ead3b4e70b1014825b3a82790" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 41509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:29.203408 30475 consensus_queue.cc:1035] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af046522b683455f8989be6e327b0099" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:29.204593 30338 heartbeater.cc:503] Master 127.21.54.189:42221 was elected leader, sending a full tablet report...
I20260811 17:14:29.205577 30436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.205220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.129:34237)
0811 17:14:29.205420 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.205485 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260811 17:14:29.205709 30438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.205419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.131:33975)
0811 17:14:29.205586 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.205643 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 17:14:29.208742 30479 mvcc.cc:205] Tried to move back new op lower bound from 7317374849847762944 to 7317374849759965184. Current Snapshot: MvccSnapshot[applied={T|T < 7317374849847762944}]
I20260811 17:14:29.211318 30438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.210943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.131:44567)
0811 17:14:29.211171 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.211233 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260811 17:14:29.211813 30436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.210116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.129:49445)
0811 17:14:29.211682 (+ 1566us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.211741 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1491}
I20260811 17:14:29.213150 30437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.209772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.130:44369)
0811 17:14:29.212996 (+ 3224us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.213052 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3164}
I20260811 17:14:29.217831 30437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.217089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38223 (local address 127.21.54.130:57355)
0811 17:14:29.217682 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.217746 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":513}
I20260811 17:14:29.218485 29568 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:29.218668 29568 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:29.222985 29568 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e72959b06b32490fb14d04885db83c44: Sending DeleteTablet for 3 replicas of tablet 43953cf9bfa84acea4fe4ab92892e9ed
I20260811 17:14:29.224017 30256 tablet_service.cc:1514] Processing DeleteTablet for tablet 43953cf9bfa84acea4fe4ab92892e9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:29 UTC) from {username='oryx', principal='oryx/127.21.54.189@KRBTEST.COM'} at 127.0.0.1:48614
I20260811 17:14:29.224010 29925 tablet_service.cc:1514] Processing DeleteTablet for tablet 43953cf9bfa84acea4fe4ab92892e9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:29 UTC) from {username='oryx', principal='oryx/127.21.54.189@KRBTEST.COM'} at 127.0.0.1:33558
I20260811 17:14:29.224444 30493 tablet_replica.cc:330] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790: stopping tablet replica
I20260811 17:14:29.224577 30090 tablet_service.cc:1514] Processing DeleteTablet for tablet 43953cf9bfa84acea4fe4ab92892e9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:29 UTC) from {username='oryx', principal='oryx/127.21.54.189@KRBTEST.COM'} at 127.0.0.1:59602
I20260811 17:14:29.224627 30493 raft_consensus.cc:2229] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:29.224771 30493 raft_consensus.cc:2258] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:29.225579 30493 ts_tablet_manager.cc:1905] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:29.225741 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 29739
I20260811 17:14:29.229161 30493 ts_tablet_manager.cc:1918] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:14:29.229272 30493 log.cc:1196] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/43953cf9bfa84acea4fe4ab92892e9ed
I20260811 17:14:29.229562 30493 ts_tablet_manager.cc:1939] T 43953cf9bfa84acea4fe4ab92892e9ed P 23f3969ead3b4e70b1014825b3a82790: Deleting consensus metadata
I20260811 17:14:29.230661 29551 catalog_manager.cc:4841] TS 23f3969ead3b4e70b1014825b3a82790 (127.21.54.131:41509): tablet 43953cf9bfa84acea4fe4ab92892e9ed (table test-table [id=73071370a13546b7a1e6c0ad84b94012]) successfully deleted
I20260811 17:14:29.235034 30494 tablet_replica.cc:330] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1: stopping tablet replica
I20260811 17:14:29.235193 30494 raft_consensus.cc:2229] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:14:29.235574 30494 raft_consensus.cc:2258] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:29.236418 30494 ts_tablet_manager.cc:1905] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:29.239327 30494 ts_tablet_manager.cc:1918] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:14:29.239424 30494 log.cc:1196] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/43953cf9bfa84acea4fe4ab92892e9ed
I20260811 17:14:29.239729 30494 ts_tablet_manager.cc:1939] T 43953cf9bfa84acea4fe4ab92892e9ed P 6127ec870149479e80c2b8d80e1e20a1: Deleting consensus metadata
I20260811 17:14:29.241022 29552 catalog_manager.cc:4841] TS 6127ec870149479e80c2b8d80e1e20a1 (127.21.54.130:42439): tablet 43953cf9bfa84acea4fe4ab92892e9ed (table test-table [id=73071370a13546b7a1e6c0ad84b94012]) successfully deleted
I20260811 17:14:29.244304 30462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:29.243997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:37479 (local address 127.0.0.1:33574)
0811 17:14:29.244142 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:29.244206 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.129:37479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 17:14:29.244293 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 30016
W20260811 17:14:29.248359 29552 catalog_manager.cc:4584] TS af046522b683455f8989be6e327b0099 (127.21.54.129:37479): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 43953cf9bfa84acea4fe4ab92892e9ed: Network error: Client connection negotiation failed: client connection to 127.21.54.129:37479: connect: Connection refused (error 111)
I20260811 17:14:29.254882 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 30179
I20260811 17:14:29.263865 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 29406
I20260811 17:14:29.275344 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 29524
2026-08-11T17:14:29Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7321 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:14:29 dist-test-slave-fd7l krb5kdc[30498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:14:29 dist-test-slave-fd7l krb5kdc[30498](info): set up 2 sockets
Aug 11 17:14:29 dist-test-slave-fd7l krb5kdc[30498](info): commencing operation
krb5kdc: starting...
W20260811 17:14:30.435684 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:14:30 dist-test-slave-fd7l krb5kdc[30498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:14:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:14:30Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.190.keytab.
Entry for principal oryx/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.190.keytab.
I20260811 17:14:30.576736 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:42881
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.190.keytab
--principal=oryx/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:42881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:30.694718 30512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:30.695057 30512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:30.695137 30512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:30.699273 30512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:30.699389 30512 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 17:14:30.699431 30512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:30.699473 30512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:30.699512 30512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:30.699558 30512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:14:30.705380 30512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:42881
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.190.keytab
--principal=oryx/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:42881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:30.706686 30512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:30.707800 30512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:30.716885 30520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:30.716877 30523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:30.717002 30512 server_base.cc:1034] running on GCE node
W20260811 17:14:30.716888 30521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:30.717841 30512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:30.719010 30512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:30.720220 30512 hybrid_clock.cc:648] HybridClock initialized: now 1786468470720181 us; error 51 us; skew 500 ppm
Aug 11 17:14:30 dist-test-slave-fd7l krb5kdc[30498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468470, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:30.723402 30512 init.cc:377] Logged in from keytab as oryx/127.21.54.190@KRBTEST.COM (short username oryx)
I20260811 17:14:30.724383 30512 webserver.cc:460] Webserver started at http://127.21.54.190:34465/ using document root <none> and password file <none>
I20260811 17:14:30.725040 30512 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:30.725112 30512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:30.725302 30512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:30.727452 30512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "6a4c1bf54035469295770754b7bf6fcc"
format_stamp: "Formatted at 2026-08-11 17:14:30 on dist-test-slave-fd7l"
server_key: "4b6183790d0459b0ac30d71f4ebb678c"
server_key_iv: "132f530365e595c02d2bc9ade7426b03"
server_key_version: "encryptionkey@0"
I20260811 17:14:30.728019 30512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "6a4c1bf54035469295770754b7bf6fcc"
format_stamp: "Formatted at 2026-08-11 17:14:30 on dist-test-slave-fd7l"
server_key: "4b6183790d0459b0ac30d71f4ebb678c"
server_key_iv: "132f530365e595c02d2bc9ade7426b03"
server_key_version: "encryptionkey@0"
I20260811 17:14:30.732115 30512 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:14:30.734997 30533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:30.736419 30512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 17:14:30.736667 30512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "6a4c1bf54035469295770754b7bf6fcc"
format_stamp: "Formatted at 2026-08-11 17:14:30 on dist-test-slave-fd7l"
server_key: "4b6183790d0459b0ac30d71f4ebb678c"
server_key_iv: "132f530365e595c02d2bc9ade7426b03"
server_key_version: "encryptionkey@0"
I20260811 17:14:30.736828 30512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:30.764079 30512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:30.767694 30512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:30.767938 30512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:30.777060 30512 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:42881
I20260811 17:14:30.777091 30606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:42881 every 8 connection(s)
I20260811 17:14:30.778136 30512 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:14:30.781421 30607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:30.785018 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 30512
I20260811 17:14:30.785141 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:14:30.785461 21722 external_mini_cluster.cc:1428] Setting key 614ba953272e739a861afd3564914da6
I20260811 17:14:30.788344 30607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc: Bootstrap starting.
I20260811 17:14:30.791165 30607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:30.792109 30607 log.cc:824] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:30.794342 30607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc: No bootstrap required, opened a new log
Aug 11 17:14:30 dist-test-slave-fd7l krb5kdc[30498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM
I20260811 17:14:30.798362 30607 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4c1bf54035469295770754b7bf6fcc" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42881 } }
I20260811 17:14:30.798655 30607 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:30.798743 30607 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4c1bf54035469295770754b7bf6fcc, State: Initialized, Role: FOLLOWER
I20260811 17:14:30.799290 30607 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4c1bf54035469295770754b7bf6fcc" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42881 } }
I20260811 17:14:30.799480 30607 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:14:30.799563 30607 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:14:30.799664 30607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:30.800848 30607 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4c1bf54035469295770754b7bf6fcc" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42881 } }
I20260811 17:14:30.801235 30611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:30.787061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51182 (local address 127.21.54.190:42881)
0811 17:14:30.787523 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:30.787540 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:30.787585 (+ 45us) server_negotiation.cc:407] Connection header received
0811 17:14:30.788215 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:30.788232 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:30.788672 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:30.788969 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:30.789974 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:30.790986 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:30.792254 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:30.792868 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:30.793069 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:30.795762 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:30.795773 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:30.795788 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:30.795819 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:30.797768 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:30.798291 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:30.798298 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:30.798304 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:30.798450 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:30.798813 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:30.798816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:30.798818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:30.799367 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:30.799612 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:30.799964 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 17:14:30.800306 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":149,"threads_started":1}
I20260811 17:14:30.801222 30607 leader_election.cc:302] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a4c1bf54035469295770754b7bf6fcc; no voters:
I20260811 17:14:30.801684 30607 leader_election.cc:288] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:14:30.801765 30613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:30.801978 30613 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc [term 1 LEADER]: Becoming Leader. State: Replica: 6a4c1bf54035469295770754b7bf6fcc, State: Running, Role: LEADER
I20260811 17:14:30.802307 30613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4c1bf54035469295770754b7bf6fcc" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42881 } }
I20260811 17:14:30.803072 30607 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:30.803941 30614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a4c1bf54035469295770754b7bf6fcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4c1bf54035469295770754b7bf6fcc" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42881 } } }
I20260811 17:14:30.804104 30614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:30.804351 30614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a4c1bf54035469295770754b7bf6fcc. Latest consensus state: current_term: 1 leader_uuid: "6a4c1bf54035469295770754b7bf6fcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4c1bf54035469295770754b7bf6fcc" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42881 } } }
I20260811 17:14:30.804458 30614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:30.804499 30625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:14:30.807545 30625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:14:30.814224 30625 catalog_manager.cc:1347] Generated new cluster ID: ccb9071ce6d84eb88211a82801cc1de5
I20260811 17:14:30.814317 30625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:14:30.821596 30625 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:14:30.822626 30625 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:14:30.828619 30625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc: Generated new TSK 0
I20260811 17:14:30.829440 30625 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.129.keytab.
Entry for principal oryx/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.129.keytab.
I20260811 17:14:30.866299 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:42881
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.129.keytab
--principal=oryx/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:30.989405 30639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:30.989710 30639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:30.989758 30639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:30.994004 30639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:30.994069 30639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:30.994113 30639 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 17:14:30.994182 30639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:14:30.999621 30639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.129.keytab
--principal=oryx/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.54.190:42881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:31.000869 30639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:31.001945 30639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:31.013198 30646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:31.013198 30645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:31.013329 30639 server_base.cc:1034] running on GCE node
W20260811 17:14:31.013226 30648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:31.013866 30639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:31.014554 30639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:31.015770 30639 hybrid_clock.cc:648] HybridClock initialized: now 1786468471015747 us; error 36 us; skew 500 ppm
Aug 11 17:14:31 dist-test-slave-fd7l krb5kdc[30498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468471, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:31.023777 30639 init.cc:377] Logged in from keytab as oryx/127.21.54.129@KRBTEST.COM (short username oryx)
I20260811 17:14:31.024991 30639 webserver.cc:460] Webserver started at http://127.21.54.129:45347/ using document root <none> and password file <none>
I20260811 17:14:31.025624 30639 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:31.025722 30639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:31.025979 30639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:31.028360 30639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "57339eab846a4db1a9c7980db2605dfa"
format_stamp: "Formatted at 2026-08-11 17:14:31 on dist-test-slave-fd7l"
server_key: "0e3f85138fb9fdc9e8878d24a54940fe"
server_key_iv: "959f3813c5158f1943d6d43cb959a612"
server_key_version: "encryptionkey@0"
I20260811 17:14:31.029066 30639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "57339eab846a4db1a9c7980db2605dfa"
format_stamp: "Formatted at 2026-08-11 17:14:31 on dist-test-slave-fd7l"
server_key: "0e3f85138fb9fdc9e8878d24a54940fe"
server_key_iv: "959f3813c5158f1943d6d43cb959a612"
server_key_version: "encryptionkey@0"
I20260811 17:14:31.033198 30639 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:14:31.035986 30655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:31.037372 30639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:31.037529 30639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "57339eab846a4db1a9c7980db2605dfa"
format_stamp: "Formatted at 2026-08-11 17:14:31 on dist-test-slave-fd7l"
server_key: "0e3f85138fb9fdc9e8878d24a54940fe"
server_key_iv: "959f3813c5158f1943d6d43cb959a612"
server_key_version: "encryptionkey@0"
I20260811 17:14:31.037654 30639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:31.052284 30639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:31.056085 30639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:31.056300 30639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:31.057899 30639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:31.057977 30639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:31.058059 30639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:31.058104 30639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 17:14:31 dist-test-slave-fd7l krb5kdc[30498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468471, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM
I20260811 17:14:31.069939 30639 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:35617
I20260811 17:14:31.070107 30796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:35617 every 8 connection(s)
I20260811 17:14:31.070937 30639 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:14:31.073369 30611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.059112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:50093 (local address 127.21.54.190:42881)
0811 17:14:31.059251 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:31.059255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:31.060384 (+ 1129us) server_negotiation.cc:407] Connection header received
0811 17:14:31.061661 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:31.061667 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:31.061736 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:31.061838 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:31.063280 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.063952 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.065308 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.065624 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.065750 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.069236 (+ 3486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:31.069239 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:31.069241 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:31.069270 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:31.071072 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:31.071746 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:31.071750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:31.071752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:31.071856 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:31.072231 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:31.072234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:31.072235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:31.072428 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:31.072554 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:31.073106 (+ 552us) server_negotiation.cc:299] Negotiation successful
0811 17:14:31.073242 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 17:14:31.074209 30664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.059683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:42881 (local address 127.21.54.129:50093)
0811 17:14:31.060211 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:31.060257 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:31.061370 (+ 1113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:31.062028 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:31.062035 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:31.062416 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:31.063111 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.063127 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.064113 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.064116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.065172 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.065183 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.065810 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.065814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.066002 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.066745 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:31.066766 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:31.069025 (+ 2259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:31.071218 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:31.071225 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:31.071241 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:31.071620 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:31.071974 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:31.071978 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:31.071980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:31.072149 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:31.072562 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:31.072574 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:31.072821 (+ 247us) client_negotiation.cc:769] Sending connection context
0811 17:14:31.073022 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 17:14:31.073294 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":131,"threads_started":1}
I20260811 17:14:31.075412 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 30639
I20260811 17:14:31.075551 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:14:31.075812 30797 heartbeater.cc:347] Connected to a master server at 127.21.54.190:42881
I20260811 17:14:31.075932 21722 external_mini_cluster.cc:1428] Setting key 2415af39a593d7e3c2ada70e8f636ad4
I20260811 17:14:31.075981 30797 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:31.076651 30797 heartbeater.cc:511] Master 127.21.54.190:42881 requested a full tablet report, sending...
I20260811 17:14:31.078373 30548 ts_manager.cc:194] Registered new tserver with Master: 57339eab846a4db1a9c7980db2605dfa (127.21.54.129:35617)
I20260811 17:14:31.080016 30548 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:50093
I20260811 17:14:31.087529 30611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.078270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:51929 (local address 127.21.54.190:42881)
0811 17:14:31.078391 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:31.078396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:31.078656 (+ 260us) server_negotiation.cc:407] Connection header received
0811 17:14:31.078791 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:31.078794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:31.078844 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:31.078931 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:31.079387 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.080123 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.081417 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.081709 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.081809 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.083454 (+ 1645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:31.083457 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:31.083459 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:31.083484 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:31.085227 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:31.085948 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:31.085952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:31.085953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:31.086063 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:31.086524 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:31.086527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:31.086528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:31.086747 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:31.086842 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:31.087251 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 17:14:31.087375 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":51}
I20260811 17:14:31.087630 30804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.078212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:42881 (local address 127.21.54.129:51929)
0811 17:14:31.078558 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:31.078571 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:31.078699 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:31.078946 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:31.078950 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:31.079170 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:31.079270 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.079276 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.080292 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.080297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.081225 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.081236 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.081849 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.081853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.081955 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.082706 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:31.082723 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:31.083275 (+ 552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:31.085366 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:31.085373 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:31.085377 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:31.085675 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:31.086191 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:31.086196 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:31.086201 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:31.086325 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:31.086864 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:31.086870 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:31.087028 (+ 158us) client_negotiation.cc:769] Sending connection context
0811 17:14:31.087162 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 17:14:31.087350 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for oryx/127.21.54.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.54.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.130.keytab.
Entry for principal oryx/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.130.keytab.
I20260811 17:14:31.112936 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:42881
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.130.keytab
--principal=oryx/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:31.231421 30809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:31.231766 30809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:31.231856 30809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:31.235946 30809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:31.236052 30809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:31.236132 30809 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 17:14:31.236214 30809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:14:31.242277 30809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.130.keytab
--principal=oryx/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.54.190:42881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:31.243462 30809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:31.244601 30809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:31.254760 30816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:31.254760 30818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:31.254782 30815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:31.254842 30809 server_base.cc:1034] running on GCE node
I20260811 17:14:31.255479 30809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:31.256174 30809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:31.257382 30809 hybrid_clock.cc:648] HybridClock initialized: now 1786468471257365 us; error 27 us; skew 500 ppm
Aug 11 17:14:31 dist-test-slave-fd7l krb5kdc[30498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468471, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:31.260317 30809 init.cc:377] Logged in from keytab as oryx/127.21.54.130@KRBTEST.COM (short username oryx)
I20260811 17:14:31.261413 30809 webserver.cc:460] Webserver started at http://127.21.54.130:37751/ using document root <none> and password file <none>
I20260811 17:14:31.262039 30809 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:31.262133 30809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:31.262403 30809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:31.264616 30809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b"
format_stamp: "Formatted at 2026-08-11 17:14:31 on dist-test-slave-fd7l"
server_key: "474bc9c2c8855a04076c579f6709365a"
server_key_iv: "f265668e8b105f22786b72abefb0b9bc"
server_key_version: "encryptionkey@0"
I20260811 17:14:31.265234 30809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b"
format_stamp: "Formatted at 2026-08-11 17:14:31 on dist-test-slave-fd7l"
server_key: "474bc9c2c8855a04076c579f6709365a"
server_key_iv: "f265668e8b105f22786b72abefb0b9bc"
server_key_version: "encryptionkey@0"
I20260811 17:14:31.269282 30809 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 17:14:31.271989 30825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:31.273572 30809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 17:14:31.273747 30809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b"
format_stamp: "Formatted at 2026-08-11 17:14:31 on dist-test-slave-fd7l"
server_key: "474bc9c2c8855a04076c579f6709365a"
server_key_iv: "f265668e8b105f22786b72abefb0b9bc"
server_key_version: "encryptionkey@0"
I20260811 17:14:31.273882 30809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:31.297976 30809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:31.301728 30809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:31.301967 30809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:31.303678 30809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:31.303752 30809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:31.303822 30809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:31.303872 30809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 17:14:31 dist-test-slave-fd7l krb5kdc[30498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468471, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM
I20260811 17:14:31.316043 30809 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:36755
I20260811 17:14:31.316097 30963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:36755 every 8 connection(s)
I20260811 17:14:31.317272 30809 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:14:31.318966 30611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.304568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:42437 (local address 127.21.54.190:42881)
0811 17:14:31.304726 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:31.304732 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:31.305842 (+ 1110us) server_negotiation.cc:407] Connection header received
0811 17:14:31.306932 (+ 1090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:31.306936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:31.307003 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:31.307092 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:31.308934 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.309678 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.310870 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.311167 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.311291 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.314352 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:31.314359 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:31.314364 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:31.314401 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:31.316359 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:31.317017 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:31.317020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:31.317021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:31.317110 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:31.317645 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:31.317649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:31.317650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:31.317958 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:31.318076 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:31.318722 (+ 646us) server_negotiation.cc:299] Negotiation successful
0811 17:14:31.318841 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 17:14:31.319839 30836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.305186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:42881 (local address 127.21.54.130:42437)
0811 17:14:31.305684 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:31.305727 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:31.306656 (+ 929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:31.307583 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:31.307590 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:31.308009 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:31.308761 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.308785 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.309815 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.309819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.310732 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.310743 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.311368 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.311372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.311570 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.312255 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:31.312273 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:31.314109 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:31.316483 (+ 2374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:31.316489 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:31.316504 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:31.316886 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:31.317238 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:31.317244 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:31.317250 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:31.317499 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:31.318135 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:31.318142 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:31.318447 (+ 305us) client_negotiation.cc:769] Sending connection context
0811 17:14:31.318620 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 17:14:31.318888 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":128,"threads_started":1}
I20260811 17:14:31.321241 30964 heartbeater.cc:347] Connected to a master server at 127.21.54.190:42881
I20260811 17:14:31.321414 30964 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:31.321933 30964 heartbeater.cc:511] Master 127.21.54.190:42881 requested a full tablet report, sending...
I20260811 17:14:31.322281 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 30809
I20260811 17:14:31.322400 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:14:31.322732 21722 external_mini_cluster.cc:1428] Setting key 6d61e3e8e2af702e2d467db54d231c70
I20260811 17:14:31.323163 30552 ts_manager.cc:194] Registered new tserver with Master: 2cba9eb13c9442a4a4bd398dcd24aa8b (127.21.54.130:36755)
I20260811 17:14:31.323843 30552 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:42437
I20260811 17:14:31.332412 30611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.324102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:48901 (local address 127.21.54.190:42881)
0811 17:14:31.324214 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:31.324218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:31.324228 (+ 10us) server_negotiation.cc:407] Connection header received
0811 17:14:31.324256 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:31.324257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:31.324289 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:31.324380 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:31.324945 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.325789 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.326642 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.326863 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.326998 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.328319 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:31.328326 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:31.328332 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:31.328371 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:31.330222 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:31.330874 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:31.330879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:31.330884 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:31.330999 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:31.331277 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:31.331283 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:31.331288 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:31.331572 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:31.331658 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:31.332172 (+ 514us) server_negotiation.cc:299] Negotiation successful
0811 17:14:31.332285 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 17:14:31.332576 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.323691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:42881 (local address 127.21.54.130:48901)
0811 17:14:31.323993 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:31.324009 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:31.324105 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:31.324398 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:31.324400 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:31.324687 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:31.324805 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.324813 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.325948 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.325953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.326512 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.326520 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.327015 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.327018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.327109 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.327727 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:31.327743 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:31.328176 (+ 433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:31.330353 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:31.330356 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:31.330358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:31.330729 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:31.331098 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:31.331100 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:31.331102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:31.331183 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:31.331711 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:31.331722 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:31.331880 (+ 158us) client_negotiation.cc:769] Sending connection context
0811 17:14:31.332065 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 17:14:31.332276 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for oryx/127.21.54.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.54.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.131.keytab.
Entry for principal oryx/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.131.keytab.
I20260811 17:14:31.356242 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:42881
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.131.keytab
--principal=oryx/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:31.481187 30974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:31.481506 30974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:31.481554 30974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:31.485888 30974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:31.485960 30974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:31.486003 30974 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 17:14:31.486037 30974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:14:31.491595 30974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.131.keytab
--principal=oryx/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.54.190:42881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:31.492769 30974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:31.493815 30974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:31.504069 30980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:31.504062 30983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:31.504062 30981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:31.504439 30974 server_base.cc:1034] running on GCE node
I20260811 17:14:31.504981 30974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:31.505611 30974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:31.506841 30974 hybrid_clock.cc:648] HybridClock initialized: now 1786468471506818 us; error 32 us; skew 500 ppm
Aug 11 17:14:31 dist-test-slave-fd7l krb5kdc[30498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468471, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:31.510237 30974 init.cc:377] Logged in from keytab as oryx/127.21.54.131@KRBTEST.COM (short username oryx)
I20260811 17:14:31.511245 30974 webserver.cc:460] Webserver started at http://127.21.54.131:42377/ using document root <none> and password file <none>
I20260811 17:14:31.511847 30974 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:31.511912 30974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:31.512147 30974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:31.514514 30974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "fb61ec037ba54c36a368da1db9daa2d9"
format_stamp: "Formatted at 2026-08-11 17:14:31 on dist-test-slave-fd7l"
server_key: "faa1a959a9f917b4b2debc50c15a1741"
server_key_iv: "c1a32c487d5fa546728b22bba91e586a"
server_key_version: "encryptionkey@0"
I20260811 17:14:31.515110 30974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "fb61ec037ba54c36a368da1db9daa2d9"
format_stamp: "Formatted at 2026-08-11 17:14:31 on dist-test-slave-fd7l"
server_key: "faa1a959a9f917b4b2debc50c15a1741"
server_key_iv: "c1a32c487d5fa546728b22bba91e586a"
server_key_version: "encryptionkey@0"
I20260811 17:14:31.519841 30974 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:14:31.523038 30992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:31.524631 30974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:31.524791 30974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "fb61ec037ba54c36a368da1db9daa2d9"
format_stamp: "Formatted at 2026-08-11 17:14:31 on dist-test-slave-fd7l"
server_key: "faa1a959a9f917b4b2debc50c15a1741"
server_key_iv: "c1a32c487d5fa546728b22bba91e586a"
server_key_version: "encryptionkey@0"
I20260811 17:14:31.524927 30974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:31.549623 30974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:31.553514 30974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:31.553752 30974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:31.555452 30974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:31.555532 30974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:31.555614 30974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:31.555665 30974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 17:14:31 dist-test-slave-fd7l krb5kdc[30498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468471, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM
I20260811 17:14:31.567294 30974 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:45139
I20260811 17:14:31.567366 31129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:45139 every 8 connection(s)
I20260811 17:14:31.568567 30974 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:14:31.571029 30611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.556671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:41507 (local address 127.21.54.190:42881)
0811 17:14:31.556830 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:31.556834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:31.558018 (+ 1184us) server_negotiation.cc:407] Connection header received
0811 17:14:31.559403 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:31.559408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:31.559482 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:31.559599 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:31.561284 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.561844 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.563327 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.563567 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.563698 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.566980 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:31.566984 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:31.566989 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:31.567024 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:31.568722 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:31.569364 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:31.569367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:31.569369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:31.569473 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:31.569850 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:31.569853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:31.569855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:31.570065 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:31.570178 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:31.570701 (+ 523us) server_negotiation.cc:299] Negotiation successful
0811 17:14:31.570842 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 17:14:31.571872 31003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.557240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:42881 (local address 127.21.54.131:41507)
0811 17:14:31.557829 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:31.557874 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:31.559102 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:31.559845 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:31.559854 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:31.560398 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:31.561106 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.561123 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.562000 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.562004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.563171 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.563185 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.563745 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.563749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.563944 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.564753 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:31.564770 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:31.566777 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:31.568848 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:31.568854 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:31.568869 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:31.569230 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:31.569577 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:31.569581 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:31.569583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:31.569737 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:31.570187 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:31.570196 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:31.570448 (+ 252us) client_negotiation.cc:769] Sending connection context
0811 17:14:31.570643 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 17:14:31.570984 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":155,"threads_started":1}
I20260811 17:14:31.573453 31130 heartbeater.cc:347] Connected to a master server at 127.21.54.190:42881
I20260811 17:14:31.573652 31130 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:31.574263 31130 heartbeater.cc:511] Master 127.21.54.190:42881 requested a full tablet report, sending...
I20260811 17:14:31.575233 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 30974
I20260811 17:14:31.575371 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:14:31.575670 30552 ts_manager.cc:194] Registered new tserver with Master: fb61ec037ba54c36a368da1db9daa2d9 (127.21.54.131:45139)
I20260811 17:14:31.575819 21722 external_mini_cluster.cc:1428] Setting key d08b837383d33d9e98f4967aeb703d6b
I20260811 17:14:31.576538 30552 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:41507
I20260811 17:14:31.578112 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:14:31.587307 31137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.575857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:42881 (local address 127.21.54.131:52891)
0811 17:14:31.576237 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:31.576254 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:31.576358 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:31.576794 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:31.576800 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:31.577045 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:31.577189 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.577196 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.580697 (+ 3501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.580701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.581284 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.581293 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.582055 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.582062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.582217 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.582786 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:31.582807 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:31.583322 (+ 515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:31.585384 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:31.585389 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:31.585392 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:31.585620 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:31.586100 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:31.586104 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:31.586106 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:31.586220 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:31.586778 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:31.586784 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:31.586914 (+ 130us) client_negotiation.cc:769] Sending connection context
0811 17:14:31.587010 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 17:14:31.587146 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":110,"threads_started":1}
I20260811 17:14:31.587491 30611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.575942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:52891 (local address 127.21.54.190:42881)
0811 17:14:31.576055 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:31.576060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:31.576320 (+ 260us) server_negotiation.cc:407] Connection header received
0811 17:14:31.576442 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:31.576445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:31.576567 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:31.576656 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:31.577358 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.578083 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.581428 (+ 3345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.581807 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.582004 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.583501 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:31.583585 (+ 84us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:31.583589 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:31.583622 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:31.585251 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:31.585729 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:31.585733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:31.585735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:31.585826 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:31.586343 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:31.586346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:31.586348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:31.586623 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:31.586749 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:31.587106 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 17:14:31.587257 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 17:14:31 dist-test-slave-fd7l krb5kdc[30498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468471, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 17:14:31 dist-test-slave-fd7l krb5kdc[30498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:31 dist-test-slave-fd7l krb5kdc[30498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 17:14:31.609395 30611 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:31.601759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51196 (local address 127.21.54.190:42881)
0811 17:14:31.601901 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:31.601906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:31.601926 (+ 20us) server_negotiation.cc:407] Connection header received
0811 17:14:31.601972 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:31.601974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:31.602015 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:31.602088 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:31.603034 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.603753 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.604886 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.605224 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.605324 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.609180 (+ 3856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51196: BlockingRecv error: recv got EOF from 127.0.0.1:51196 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 17:14:31 dist-test-slave-fd7l krb5kdc[30498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468471, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM
I20260811 17:14:31.619973 30611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.611092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51204 (local address 127.21.54.190:42881)
0811 17:14:31.611221 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:31.611224 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:31.611362 (+ 138us) server_negotiation.cc:407] Connection header received
0811 17:14:31.611542 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:31.611545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:31.611593 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:31.611671 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:31.612282 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.612899 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.613737 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.613945 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.614034 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.616623 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:31.616626 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:31.616628 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:31.616666 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:31.618247 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:31.618752 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:31.618754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:31.618755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:31.618842 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:31.619120 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:31.619122 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:31.619123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:31.619322 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:31.619431 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:31.619739 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 17:14:31.619846 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 17:14:31.623703 30552 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:14:31.626921 30552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:14:31.642881 31158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.636488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:45139 (local address 127.0.0.1:48784)
0811 17:14:31.637334 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:31.637391 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:31.637544 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:31.638440 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:31.638443 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:31.638458 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:31.638655 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.638665 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.639683 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.639689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.641058 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.641069 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.642262 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.642266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.642397 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.642443 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 17:14:31.642567 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 17:14:31.642702 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"mutex_wait_us":59,"thread_start_us":83,"threads_started":1}
I20260811 17:14:31.643061 31160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.636694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37038 (local address 127.21.54.130:36755)
0811 17:14:31.637135 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:31.637149 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:31.637515 (+ 366us) server_negotiation.cc:407] Connection header received
0811 17:14:31.637774 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:31.637780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:31.638014 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:31.638150 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:31.638752 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.639970 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.641628 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.642425 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.642545 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.642710 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:31.642814 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 17:14:31.642905 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":95,"threads_started":1}
I20260811 17:14:31.643244 31163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.637332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48784 (local address 127.21.54.131:45139)
0811 17:14:31.637929 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:31.637941 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:31.637965 (+ 24us) server_negotiation.cc:407] Connection header received
0811 17:14:31.638041 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:31.638045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:31.638241 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:31.638390 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:31.638794 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.639559 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.641266 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.642131 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.642260 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.642493 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:31.642987 (+ 494us) server_negotiation.cc:299] Negotiation successful
0811 17:14:31.643095 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":91,"threads_started":1}
I20260811 17:14:31.643234 31157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.636752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:35617 (local address 127.0.0.1:38214)
0811 17:14:31.637791 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:31.637805 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:31.637882 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:31.638509 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:31.638515 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:31.638528 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:31.638643 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.638648 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.640143 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.640146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.641584 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.641598 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.642868 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.642871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.642965 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.642987 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 17:14:31.643058 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 17:14:31.643118 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":978,"mutex_wait_us":2,"thread_start_us":99,"threads_started":1}
I20260811 17:14:31.643602 31162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.637332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38214 (local address 127.21.54.129:35617)
0811 17:14:31.637712 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:31.637719 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:31.637989 (+ 270us) server_negotiation.cc:407] Connection header received
0811 17:14:31.638073 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:31.638082 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:31.638255 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:31.638389 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:31.638806 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.639970 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.641721 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.642556 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.642657 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.642799 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:31.643347 (+ 548us) server_negotiation.cc:299] Negotiation successful
0811 17:14:31.643477 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":88,"threads_started":1}
I20260811 17:14:31.643707 31161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.636543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:36755 (local address 127.0.0.1:37038)
0811 17:14:31.637334 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:31.637391 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:31.637578 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:31.638356 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:31.638362 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:31.638386 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:31.638605 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.638616 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.640098 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.640101 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.641494 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.641504 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.642580 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.642586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.642680 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.642702 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 17:14:31.642783 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 17:14:31.642861 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"mutex_wait_us":299,"thread_start_us":126,"threads_started":1}
I20260811 17:14:31.646566 31054 tablet_service.cc:1467] Processing CreateTablet for tablet 9b87e02f6bd84d1b9a70c1078be1bbff (DEFAULT_TABLE table=test-table [id=2357608d7e4d4b92858f327b61087103]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:31.646657 30885 tablet_service.cc:1467] Processing CreateTablet for tablet 9b87e02f6bd84d1b9a70c1078be1bbff (DEFAULT_TABLE table=test-table [id=2357608d7e4d4b92858f327b61087103]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:31.646646 30718 tablet_service.cc:1467] Processing CreateTablet for tablet 9b87e02f6bd84d1b9a70c1078be1bbff (DEFAULT_TABLE table=test-table [id=2357608d7e4d4b92858f327b61087103]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:31.647639 31054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b87e02f6bd84d1b9a70c1078be1bbff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:31.647662 30718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b87e02f6bd84d1b9a70c1078be1bbff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:31.647662 30885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b87e02f6bd84d1b9a70c1078be1bbff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:31.655228 31164 tablet_bootstrap.cc:492] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9: Bootstrap starting.
I20260811 17:14:31.656019 31165 tablet_bootstrap.cc:492] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa: Bootstrap starting.
I20260811 17:14:31.656795 31166 tablet_bootstrap.cc:492] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b: Bootstrap starting.
I20260811 17:14:31.658417 31165 tablet_bootstrap.cc:654] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:31.658474 31166 tablet_bootstrap.cc:654] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:31.659380 31166 log.cc:824] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:31.659940 31165 log.cc:824] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:31.660823 31164 tablet_bootstrap.cc:654] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:31.661527 31164 log.cc:824] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:31.661942 31166 tablet_bootstrap.cc:492] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b: No bootstrap required, opened a new log
I20260811 17:14:31.661942 31165 tablet_bootstrap.cc:492] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa: No bootstrap required, opened a new log
I20260811 17:14:31.662168 31166 ts_tablet_manager.cc:1397] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 17:14:31.662168 31165 ts_tablet_manager.cc:1397] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 17:14:31.663307 31164 tablet_bootstrap.cc:492] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9: No bootstrap required, opened a new log
I20260811 17:14:31.663519 31164 ts_tablet_manager.cc:1397] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 17:14:31.666549 31166 raft_consensus.cc:348] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } } peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } }
I20260811 17:14:31.666740 31164 raft_consensus.cc:348] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } } peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } }
I20260811 17:14:31.666982 31166 raft_consensus.cc:374] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:31.667030 31164 raft_consensus.cc:374] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:31.666862 31165 raft_consensus.cc:348] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } } peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } }
I20260811 17:14:31.667047 31166 raft_consensus.cc:729] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cba9eb13c9442a4a4bd398dcd24aa8b, State: Initialized, Role: FOLLOWER
I20260811 17:14:31.667110 31165 raft_consensus.cc:374] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:31.667136 31164 raft_consensus.cc:729] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb61ec037ba54c36a368da1db9daa2d9, State: Initialized, Role: FOLLOWER
I20260811 17:14:31.667197 31165 raft_consensus.cc:729] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57339eab846a4db1a9c7980db2605dfa, State: Initialized, Role: FOLLOWER
I20260811 17:14:31.667618 31164 consensus_queue.cc:260] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } } peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } }
I20260811 17:14:31.667598 31166 consensus_queue.cc:260] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } } peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } }
I20260811 17:14:31.667598 31165 consensus_queue.cc:260] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } } peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } }
I20260811 17:14:31.668627 31164 ts_tablet_manager.cc:1428] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 17:14:31.668669 31166 ts_tablet_manager.cc:1428] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 17:14:31.668669 31165 ts_tablet_manager.cc:1428] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260811 17:14:31.668994 31130 heartbeater.cc:503] Master 127.21.54.190:42881 was elected leader, sending a full tablet report...
I20260811 17:14:31.669016 30797 heartbeater.cc:503] Master 127.21.54.190:42881 was elected leader, sending a full tablet report...
I20260811 17:14:31.669046 30964 heartbeater.cc:503] Master 127.21.54.190:42881 was elected leader, sending a full tablet report...
I20260811 17:14:31.761085 31173 raft_consensus.cc:482] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:14:31.761353 31173 raft_consensus.cc:504] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } } peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } }
I20260811 17:14:31.762697 31173 leader_election.cc:288] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb61ec037ba54c36a368da1db9daa2d9 (127.21.54.131:45139), 57339eab846a4db1a9c7980db2605dfa (127.21.54.129:35617)
I20260811 17:14:31.766374 30836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.762856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:45139 (local address 127.21.54.130:52343)
0811 17:14:31.763060 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:31.763078 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:31.763181 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:31.763470 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:31.763473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:31.763493 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:31.763617 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.763623 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.764432 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.764435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.765262 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.765271 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.765927 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.765931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.766041 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.766065 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 17:14:31.766156 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 17:14:31.766233 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260811 17:14:31.766376 31163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.763018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:52343 (local address 127.21.54.131:45139)
0811 17:14:31.763178 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:31.763185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:31.763206 (+ 21us) server_negotiation.cc:407] Connection header received
0811 17:14:31.763287 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:31.763290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:31.763342 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:31.763448 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:31.763747 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.764307 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.765398 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.765786 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.765927 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.765982 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:31.766175 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 17:14:31.766248 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 17:14:31.767098 31070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b87e02f6bd84d1b9a70c1078be1bbff" candidate_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb61ec037ba54c36a368da1db9daa2d9" is_pre_election: true
I20260811 17:14:31.767400 31175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.763083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:35617 (local address 127.21.54.130:57611)
0811 17:14:31.763467 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:31.763484 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:31.763606 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:31.763877 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:31.763879 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:31.763895 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:31.764027 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.764038 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.764839 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.764843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.766103 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:31.766112 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.766970 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.766974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:31.767081 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.767103 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 17:14:31.767191 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 17:14:31.767258 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":161,"threads_started":1}
I20260811 17:14:31.767392 31162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.763299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:57611 (local address 127.21.54.129:35617)
0811 17:14:31.763423 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:31.763428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:31.763573 (+ 145us) server_negotiation.cc:407] Connection header received
0811 17:14:31.763724 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:31.763727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:31.763776 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:31.763864 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:31.764166 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.764726 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.766243 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.766818 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.766968 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.767018 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:31.767209 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 17:14:31.767287 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 17:14:31.767482 31070 raft_consensus.cc:2454] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cba9eb13c9442a4a4bd398dcd24aa8b in term 0.
I20260811 17:14:31.767963 30741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b87e02f6bd84d1b9a70c1078be1bbff" candidate_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57339eab846a4db1a9c7980db2605dfa" is_pre_election: true
I20260811 17:14:31.768076 30829 leader_election.cc:302] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cba9eb13c9442a4a4bd398dcd24aa8b, fb61ec037ba54c36a368da1db9daa2d9; no voters:
I20260811 17:14:31.768242 30741 raft_consensus.cc:2454] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cba9eb13c9442a4a4bd398dcd24aa8b in term 0.
I20260811 17:14:31.768457 31173 raft_consensus.cc:2781] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:14:31.768563 31173 raft_consensus.cc:482] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:14:31.768651 31173 raft_consensus.cc:3037] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:31.769779 31173 raft_consensus.cc:504] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } } peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } }
I20260811 17:14:31.770191 31173 leader_election.cc:288] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [CANDIDATE]: Term 1 election: Requested vote from peers fb61ec037ba54c36a368da1db9daa2d9 (127.21.54.131:45139), 57339eab846a4db1a9c7980db2605dfa (127.21.54.129:35617)
I20260811 17:14:31.770576 31070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b87e02f6bd84d1b9a70c1078be1bbff" candidate_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb61ec037ba54c36a368da1db9daa2d9"
I20260811 17:14:31.770633 30741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b87e02f6bd84d1b9a70c1078be1bbff" candidate_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57339eab846a4db1a9c7980db2605dfa"
I20260811 17:14:31.770711 31070 raft_consensus.cc:3037] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:31.770787 30741 raft_consensus.cc:3037] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:31.771888 31070 raft_consensus.cc:2454] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cba9eb13c9442a4a4bd398dcd24aa8b in term 1.
I20260811 17:14:31.772182 30741 raft_consensus.cc:2454] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cba9eb13c9442a4a4bd398dcd24aa8b in term 1.
I20260811 17:14:31.772312 30829 leader_election.cc:302] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cba9eb13c9442a4a4bd398dcd24aa8b, fb61ec037ba54c36a368da1db9daa2d9; no voters:
I20260811 17:14:31.772576 31173 raft_consensus.cc:2781] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:31.772797 31173 raft_consensus.cc:686] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 1 LEADER]: Becoming Leader. State: Replica: 2cba9eb13c9442a4a4bd398dcd24aa8b, State: Running, Role: LEADER
I20260811 17:14:31.773213 31173 consensus_queue.cc:237] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } } peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } }
I20260811 17:14:31.777069 30548 catalog_manager.cc:5495] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cba9eb13c9442a4a4bd398dcd24aa8b (127.21.54.130). New cstate: current_term: 1 leader_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } health_report { overall_health: UNKNOWN } } }
I20260811 17:14:31.798810 31160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:31.795013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37040 (local address 127.21.54.130:36755)
0811 17:14:31.795162 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:31.795167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:31.795182 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:14:31.795279 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:31.795281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:31.795340 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:31.795443 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:14:31.795689 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.796265 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.797225 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:31.797477 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:31.797584 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:31.797893 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:14:31.798399 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:14:31.798559 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:31.798614 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 17:14:31.798680 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 17:14:31.805619 30741 raft_consensus.cc:1261] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa [term 1 FOLLOWER]: Refusing update from remote peer 2cba9eb13c9442a4a4bd398dcd24aa8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:31.805970 31070 raft_consensus.cc:1261] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9 [term 1 FOLLOWER]: Refusing update from remote peer 2cba9eb13c9442a4a4bd398dcd24aa8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:31.806511 31173 consensus_queue.cc:1035] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:31.806720 31178 consensus_queue.cc:1035] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:31.815806 31181 mvcc.cc:205] Tried to move back new op lower bound from 7317374860507062272 to 7317374860386234368. Current Snapshot: MvccSnapshot[applied={T|T < 7317374860507062272}]
I20260811 17:14:31.818753 31185 mvcc.cc:205] Tried to move back new op lower bound from 7317374860507062272 to 7317374860386234368. Current Snapshot: MvccSnapshot[applied={T|T < 7317374860507062272 or (T in {7317374860507062272})}]
I20260811 17:14:31.819387 31183 mvcc.cc:205] Tried to move back new op lower bound from 7317374860507062272 to 7317374860386234368. Current Snapshot: MvccSnapshot[applied={T|T < 7317374860507062272}]
I20260811 17:14:31.824672 30548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:31.824887 30548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:31.827844 30548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc: Sending DeleteTablet for 3 replicas of tablet 9b87e02f6bd84d1b9a70c1078be1bbff
I20260811 17:14:31.828908 30885 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b87e02f6bd84d1b9a70c1078be1bbff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:31 UTC) from {username='oryx', principal='oryx/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:37038
I20260811 17:14:31.828917 31054 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b87e02f6bd84d1b9a70c1078be1bbff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:31 UTC) from {username='oryx', principal='oryx/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:48784
I20260811 17:14:31.829520 31193 tablet_replica.cc:330] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9: stopping tablet replica
I20260811 17:14:31.829507 30718 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b87e02f6bd84d1b9a70c1078be1bbff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:31 UTC) from {username='oryx', principal='oryx/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:38214
I20260811 17:14:31.829917 31193 raft_consensus.cc:2229] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:31.830003 31194 tablet_replica.cc:330] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa: stopping tablet replica
I20260811 17:14:31.830324 31194 raft_consensus.cc:2229] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:31.830430 31193 raft_consensus.cc:2258] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:31.830700 31194 raft_consensus.cc:2258] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:31.830757 30548 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 17:14:31.830880 31192 tablet_replica.cc:330] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b: stopping tablet replica
W20260811 17:14:31.831204 30548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:14:31.831187 31192 raft_consensus.cc:2229] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:14:31.831585 31192 raft_consensus.cc:2258] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:31.832580 31193 ts_tablet_manager.cc:1905] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:31.833040 31194 ts_tablet_manager.cc:1905] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:31.833041 31192 ts_tablet_manager.cc:1905] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:31.835847 31194 ts_tablet_manager.cc:1918] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:14:31.835907 31193 ts_tablet_manager.cc:1918] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:14:31.836020 31194 log.cc:1196] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/wal/wals/9b87e02f6bd84d1b9a70c1078be1bbff
I20260811 17:14:31.836488 31194 ts_tablet_manager.cc:1939] T 9b87e02f6bd84d1b9a70c1078be1bbff P 57339eab846a4db1a9c7980db2605dfa: Deleting consensus metadata
I20260811 17:14:31.837497 30885 tablet_service.cc:1467] Processing CreateTablet for tablet 86d05ec3331148b3b3f0ddc83dcf33a1 (DEFAULT_TABLE table=test-table [id=2588764161f0482cabaa93aced172a84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:31.837780 30885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86d05ec3331148b3b3f0ddc83dcf33a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:31.839612 31192 ts_tablet_manager.cc:1918] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:14:31.839715 31192 log.cc:1196] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/9b87e02f6bd84d1b9a70c1078be1bbff
I20260811 17:14:31.839845 31166 tablet_bootstrap.cc:492] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b: Bootstrap starting.
I20260811 17:14:31.840072 31192 ts_tablet_manager.cc:1939] T 9b87e02f6bd84d1b9a70c1078be1bbff P 2cba9eb13c9442a4a4bd398dcd24aa8b: Deleting consensus metadata
I20260811 17:14:31.840963 31166 tablet_bootstrap.cc:654] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:31.840986 31193 log.cc:1196] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/9b87e02f6bd84d1b9a70c1078be1bbff
I20260811 17:14:31.841022 30534 catalog_manager.cc:4841] TS 57339eab846a4db1a9c7980db2605dfa (127.21.54.129:35617): tablet 9b87e02f6bd84d1b9a70c1078be1bbff (table test-table [id=2357608d7e4d4b92858f327b61087103]) successfully deleted
I20260811 17:14:31.841372 31193 ts_tablet_manager.cc:1939] T 9b87e02f6bd84d1b9a70c1078be1bbff P fb61ec037ba54c36a368da1db9daa2d9: Deleting consensus metadata
I20260811 17:14:31.841470 30718 tablet_service.cc:1467] Processing CreateTablet for tablet 86d05ec3331148b3b3f0ddc83dcf33a1 (DEFAULT_TABLE table=test-table [id=2588764161f0482cabaa93aced172a84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:31.841774 30718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86d05ec3331148b3b3f0ddc83dcf33a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:31.842504 31166 tablet_bootstrap.cc:492] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b: No bootstrap required, opened a new log
I20260811 17:14:31.842588 31166 ts_tablet_manager.cc:1397] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:14:31.843073 31166 raft_consensus.cc:348] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:31.843197 31166 raft_consensus.cc:374] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:31.843230 31166 raft_consensus.cc:729] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cba9eb13c9442a4a4bd398dcd24aa8b, State: Initialized, Role: FOLLOWER
I20260811 17:14:31.843375 31166 consensus_queue.cc:260] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:31.843657 31166 ts_tablet_manager.cc:1428] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 17:14:31.844630 30535 catalog_manager.cc:4841] TS 2cba9eb13c9442a4a4bd398dcd24aa8b (127.21.54.130:36755): tablet 9b87e02f6bd84d1b9a70c1078be1bbff (table test-table [id=2357608d7e4d4b92858f327b61087103]) successfully deleted
I20260811 17:14:31.844877 30537 catalog_manager.cc:4841] TS fb61ec037ba54c36a368da1db9daa2d9 (127.21.54.131:45139): tablet 9b87e02f6bd84d1b9a70c1078be1bbff (table test-table [id=2357608d7e4d4b92858f327b61087103]) successfully deleted
I20260811 17:14:31.845485 31054 tablet_service.cc:1467] Processing CreateTablet for tablet 86d05ec3331148b3b3f0ddc83dcf33a1 (DEFAULT_TABLE table=test-table [id=2588764161f0482cabaa93aced172a84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:31.845786 31054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86d05ec3331148b3b3f0ddc83dcf33a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:31.846611 31178 raft_consensus.cc:482] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:14:31.846719 31178 raft_consensus.cc:504] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:31.847164 31178 leader_election.cc:288] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57339eab846a4db1a9c7980db2605dfa (127.21.54.129:35617), fb61ec037ba54c36a368da1db9daa2d9 (127.21.54.131:45139)
I20260811 17:14:31.847496 30741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86d05ec3331148b3b3f0ddc83dcf33a1" candidate_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57339eab846a4db1a9c7980db2605dfa" is_pre_election: true
W20260811 17:14:31.848078 30826 leader_election.cc:341] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 57339eab846a4db1a9c7980db2605dfa (127.21.54.129:35617): Illegal state: must be running to vote when last-logged opid is not known
W20260811 17:14:31.848239 30829 leader_election.cc:341] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb61ec037ba54c36a368da1db9daa2d9 (127.21.54.131:45139): Not found: Tablet not found: 86d05ec3331148b3b3f0ddc83dcf33a1
I20260811 17:14:31.848294 30829 leader_election.cc:302] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2cba9eb13c9442a4a4bd398dcd24aa8b; no voters: 57339eab846a4db1a9c7980db2605dfa, fb61ec037ba54c36a368da1db9daa2d9
I20260811 17:14:31.848394 31164 tablet_bootstrap.cc:492] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9: Bootstrap starting.
I20260811 17:14:31.848424 31178 raft_consensus.cc:2726] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 17:14:31.849140 31165 tablet_bootstrap.cc:492] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa: Bootstrap starting.
I20260811 17:14:31.849449 31164 tablet_bootstrap.cc:654] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:31.850175 31165 tablet_bootstrap.cc:654] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:31.850703 31164 tablet_bootstrap.cc:492] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9: No bootstrap required, opened a new log
I20260811 17:14:31.850790 31164 ts_tablet_manager.cc:1397] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 17:14:31.851156 31164 raft_consensus.cc:348] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:31.851234 31164 raft_consensus.cc:374] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:31.851255 31164 raft_consensus.cc:729] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb61ec037ba54c36a368da1db9daa2d9, State: Initialized, Role: FOLLOWER
I20260811 17:14:31.851392 31164 consensus_queue.cc:260] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:31.851644 31164 ts_tablet_manager.cc:1428] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 17:14:31.852571 31165 tablet_bootstrap.cc:492] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa: No bootstrap required, opened a new log
I20260811 17:14:31.852691 31165 ts_tablet_manager.cc:1397] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 17:14:31.853389 31165 raft_consensus.cc:348] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:31.853513 31165 raft_consensus.cc:374] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:31.853564 31165 raft_consensus.cc:729] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57339eab846a4db1a9c7980db2605dfa, State: Initialized, Role: FOLLOWER
I20260811 17:14:31.853713 31165 consensus_queue.cc:260] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:31.854001 31165 ts_tablet_manager.cc:1428] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260811 17:14:32.069360 30965 tablet.cc:2369] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:14:32.073179 30799 tablet.cc:2369] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:14:32.074393 31131 tablet.cc:2369] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:14:32.301975 31174 raft_consensus.cc:482] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:14:32.302171 31174 raft_consensus.cc:504] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:32.303550 31174 leader_election.cc:288] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57339eab846a4db1a9c7980db2605dfa (127.21.54.129:35617), 2cba9eb13c9442a4a4bd398dcd24aa8b (127.21.54.130:36755)
I20260811 17:14:32.307837 31201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:32.304055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:50269 (local address 127.21.54.129:35617)
0811 17:14:32.304310 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:32.304317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:32.304335 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:14:32.304624 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:32.304626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:32.304670 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:32.304766 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:32.305117 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.305707 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.306766 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.307244 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.307374 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:32.307434 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:32.307608 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 17:14:32.307685 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":70,"threads_started":1}
I20260811 17:14:32.307852 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:32.303920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:35617 (local address 127.21.54.131:50269)
0811 17:14:32.304254 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:32.304269 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:32.304440 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:32.304776 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:32.304779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:32.304796 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:32.305001 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:32.305007 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.305816 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.305821 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:32.306637 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:32.306646 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.307367 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.307371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:32.307466 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:32.307488 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 17:14:32.307593 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 17:14:32.307670 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":22,"thread_start_us":99,"threads_started":1}
I20260811 17:14:32.308000 31200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:32.303979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:46057 (local address 127.21.54.130:36755)
0811 17:14:32.304268 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:32.304273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:32.304358 (+ 85us) server_negotiation.cc:407] Connection header received
0811 17:14:32.304609 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:32.304612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:32.304661 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:32.304767 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:32.305132 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.305734 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.306865 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.307358 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.307513 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:32.307573 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:32.307761 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 17:14:32.307845 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":79,"threads_started":1}
I20260811 17:14:32.308049 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:32.303873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:36755 (local address 127.21.54.131:46057)
0811 17:14:32.304246 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:32.304264 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:32.304431 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:32.304790 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:32.304793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:32.304836 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:32.305004 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:32.305010 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.305861 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.305867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:32.306729 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:32.306739 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.307500 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.307504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:32.307613 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:32.307644 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 17:14:32.307743 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 17:14:32.307818 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":33,"spinlock_wait_cycles":28416,"thread_start_us":91,"threads_started":1}
I20260811 17:14:32.308734 30910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86d05ec3331148b3b3f0ddc83dcf33a1" candidate_uuid: "fb61ec037ba54c36a368da1db9daa2d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" is_pre_election: true
I20260811 17:14:32.308812 30741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86d05ec3331148b3b3f0ddc83dcf33a1" candidate_uuid: "fb61ec037ba54c36a368da1db9daa2d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57339eab846a4db1a9c7980db2605dfa" is_pre_election: true
I20260811 17:14:32.308967 30741 raft_consensus.cc:2454] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb61ec037ba54c36a368da1db9daa2d9 in term 0.
I20260811 17:14:32.309062 30910 raft_consensus.cc:2454] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb61ec037ba54c36a368da1db9daa2d9 in term 0.
I20260811 17:14:32.309449 30993 leader_election.cc:302] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2cba9eb13c9442a4a4bd398dcd24aa8b, 57339eab846a4db1a9c7980db2605dfa, fb61ec037ba54c36a368da1db9daa2d9; no voters:
I20260811 17:14:32.309763 31174 raft_consensus.cc:2781] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:14:32.309837 31174 raft_consensus.cc:482] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:14:32.309891 31174 raft_consensus.cc:3037] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:32.310959 31174 raft_consensus.cc:504] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:32.311333 31174 leader_election.cc:288] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [CANDIDATE]: Term 1 election: Requested vote from peers 57339eab846a4db1a9c7980db2605dfa (127.21.54.129:35617), 2cba9eb13c9442a4a4bd398dcd24aa8b (127.21.54.130:36755)
I20260811 17:14:32.311759 30910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86d05ec3331148b3b3f0ddc83dcf33a1" candidate_uuid: "fb61ec037ba54c36a368da1db9daa2d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b"
I20260811 17:14:32.311846 30741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86d05ec3331148b3b3f0ddc83dcf33a1" candidate_uuid: "fb61ec037ba54c36a368da1db9daa2d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57339eab846a4db1a9c7980db2605dfa"
I20260811 17:14:32.311915 30910 raft_consensus.cc:3037] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:32.311985 30741 raft_consensus.cc:3037] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:32.313005 30741 raft_consensus.cc:2454] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb61ec037ba54c36a368da1db9daa2d9 in term 1.
I20260811 17:14:32.313133 30910 raft_consensus.cc:2454] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb61ec037ba54c36a368da1db9daa2d9 in term 1.
I20260811 17:14:32.313371 30993 leader_election.cc:302] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57339eab846a4db1a9c7980db2605dfa, fb61ec037ba54c36a368da1db9daa2d9; no voters:
I20260811 17:14:32.313606 31174 raft_consensus.cc:2781] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:32.313753 31174 raft_consensus.cc:686] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [term 1 LEADER]: Becoming Leader. State: Replica: fb61ec037ba54c36a368da1db9daa2d9, State: Running, Role: LEADER
I20260811 17:14:32.314014 31174 consensus_queue.cc:237] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:32.316644 30548 catalog_manager.cc:5495] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb61ec037ba54c36a368da1db9daa2d9 (127.21.54.131). New cstate: current_term: 1 leader_uuid: "fb61ec037ba54c36a368da1db9daa2d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } health_report { overall_health: HEALTHY } } }
Aug 11 17:14:32 dist-test-slave-fd7l krb5kdc[30498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468470, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.190@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM
I20260811 17:14:32.440379 31211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:32.430560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:42881 (local address 127.0.0.1:51206)
0811 17:14:32.430923 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:32.430942 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:32.431065 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:32.431330 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:32.431333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:32.431784 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:32.431887 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:32.431893 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.432761 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.432764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:32.433339 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:32.433347 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.433834 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.433837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:32.433906 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:32.434610 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:32.434624 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:32.436170 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:32.438278 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:32.438289 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:32.438295 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:32.438618 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:32.438974 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:32.438976 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:32.438978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:32.439070 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:32.439582 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:32.439594 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:32.439718 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 17:14:32.439822 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 17:14:32.440210 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":115,"threads_started":1}
I20260811 17:14:32.440768 31212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:32.430688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51206 (local address 127.21.54.190:42881)
0811 17:14:32.431036 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:32.431044 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:32.431060 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:14:32.431153 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:32.431156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:32.431197 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:32.431279 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:32.432011 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.432657 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.433452 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.433696 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.433783 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:32.436343 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:32.436346 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:32.436349 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:32.436372 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:32.438138 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:32.438779 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:32.438783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:32.438784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:32.438869 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:32.439197 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:32.439200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:32.439201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:32.439420 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:32.439520 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:32.439922 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 17:14:32.440339 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":227,"server-negotiator.queue_time_us":285,"thread_start_us":162,"threads_started":1}
I20260811 17:14:32.445937 30548 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:51206:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 17:14:32.446930 30548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:14:32.452996 30885 tablet_service.cc:1467] Processing CreateTablet for tablet ef9665d091f3448883f309d3cb013c98 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1f20ae6dd636492489d4f0d82df8e395]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 17:14:32.452996 30718 tablet_service.cc:1467] Processing CreateTablet for tablet ef9665d091f3448883f309d3cb013c98 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1f20ae6dd636492489d4f0d82df8e395]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 17:14:32.453480 30885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef9665d091f3448883f309d3cb013c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:32.453483 30718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef9665d091f3448883f309d3cb013c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:32.453624 31054 tablet_service.cc:1467] Processing CreateTablet for tablet ef9665d091f3448883f309d3cb013c98 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1f20ae6dd636492489d4f0d82df8e395]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 17:14:32.453944 31054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef9665d091f3448883f309d3cb013c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:32.456785 31216 tablet_bootstrap.cc:492] T ef9665d091f3448883f309d3cb013c98 P 57339eab846a4db1a9c7980db2605dfa: Bootstrap starting.
I20260811 17:14:32.456879 31213 tablet_bootstrap.cc:492] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b: Bootstrap starting.
I20260811 17:14:32.457144 31215 tablet_bootstrap.cc:492] T ef9665d091f3448883f309d3cb013c98 P fb61ec037ba54c36a368da1db9daa2d9: Bootstrap starting.
I20260811 17:14:32.457844 31216 tablet_bootstrap.cc:654] T ef9665d091f3448883f309d3cb013c98 P 57339eab846a4db1a9c7980db2605dfa: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:32.457947 31213 tablet_bootstrap.cc:654] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:32.458146 31215 tablet_bootstrap.cc:654] T ef9665d091f3448883f309d3cb013c98 P fb61ec037ba54c36a368da1db9daa2d9: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:32.459452 31215 tablet_bootstrap.cc:492] T ef9665d091f3448883f309d3cb013c98 P fb61ec037ba54c36a368da1db9daa2d9: No bootstrap required, opened a new log
I20260811 17:14:32.459501 31213 tablet_bootstrap.cc:492] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b: No bootstrap required, opened a new log
I20260811 17:14:32.459574 31215 ts_tablet_manager.cc:1397] T ef9665d091f3448883f309d3cb013c98 P fb61ec037ba54c36a368da1db9daa2d9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 17:14:32.459602 31213 ts_tablet_manager.cc:1397] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:14:32.459785 31216 tablet_bootstrap.cc:492] T ef9665d091f3448883f309d3cb013c98 P 57339eab846a4db1a9c7980db2605dfa: No bootstrap required, opened a new log
I20260811 17:14:32.459899 31216 ts_tablet_manager.cc:1397] T ef9665d091f3448883f309d3cb013c98 P 57339eab846a4db1a9c7980db2605dfa: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:14:32.460070 31215 raft_consensus.cc:348] T ef9665d091f3448883f309d3cb013c98 P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:32.460197 31215 raft_consensus.cc:374] T ef9665d091f3448883f309d3cb013c98 P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:32.460147 31213 raft_consensus.cc:348] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:32.460270 31215 raft_consensus.cc:729] T ef9665d091f3448883f309d3cb013c98 P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb61ec037ba54c36a368da1db9daa2d9, State: Initialized, Role: FOLLOWER
I20260811 17:14:32.460326 31213 raft_consensus.cc:374] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:32.460384 31213 raft_consensus.cc:729] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cba9eb13c9442a4a4bd398dcd24aa8b, State: Initialized, Role: FOLLOWER
I20260811 17:14:32.460402 31216 raft_consensus.cc:348] T ef9665d091f3448883f309d3cb013c98 P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:32.460464 31215 consensus_queue.cc:260] T ef9665d091f3448883f309d3cb013c98 P fb61ec037ba54c36a368da1db9daa2d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:32.460580 31216 raft_consensus.cc:374] T ef9665d091f3448883f309d3cb013c98 P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:32.460623 31216 raft_consensus.cc:729] T ef9665d091f3448883f309d3cb013c98 P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57339eab846a4db1a9c7980db2605dfa, State: Initialized, Role: FOLLOWER
I20260811 17:14:32.460585 31213 consensus_queue.cc:260] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:32.460783 31215 ts_tablet_manager.cc:1428] T ef9665d091f3448883f309d3cb013c98 P fb61ec037ba54c36a368da1db9daa2d9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:14:32.460734 31174 tablet_replica.cc:439] T ef9665d091f3448883f309d3cb013c98 P fb61ec037ba54c36a368da1db9daa2d9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } } }
I20260811 17:14:32.460754 31216 consensus_queue.cc:260] T ef9665d091f3448883f309d3cb013c98 P 57339eab846a4db1a9c7980db2605dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:32.460865 31174 tablet_replica.cc:442] T ef9665d091f3448883f309d3cb013c98 P fb61ec037ba54c36a368da1db9daa2d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:14:32.460990 31213 ts_tablet_manager.cc:1428] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:14:32.461174 31216 ts_tablet_manager.cc:1428] T ef9665d091f3448883f309d3cb013c98 P 57339eab846a4db1a9c7980db2605dfa: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 17:14:32.461311 31221 tablet_replica.cc:439] T ef9665d091f3448883f309d3cb013c98 P 57339eab846a4db1a9c7980db2605dfa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } } }
I20260811 17:14:32.461874 31220 tablet_replica.cc:439] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } } }
I20260811 17:14:32.461979 31221 tablet_replica.cc:442] T ef9665d091f3448883f309d3cb013c98 P 57339eab846a4db1a9c7980db2605dfa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:14:32.462057 31220 tablet_replica.cc:442] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:14:32.650663 31220 raft_consensus.cc:482] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:14:32.650817 31220 raft_consensus.cc:504] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:32.651331 31220 leader_election.cc:288] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57339eab846a4db1a9c7980db2605dfa (127.21.54.129:35617), fb61ec037ba54c36a368da1db9daa2d9 (127.21.54.131:45139)
I20260811 17:14:32.651908 31070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef9665d091f3448883f309d3cb013c98" candidate_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb61ec037ba54c36a368da1db9daa2d9" is_pre_election: true
I20260811 17:14:32.651921 30741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef9665d091f3448883f309d3cb013c98" candidate_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57339eab846a4db1a9c7980db2605dfa" is_pre_election: true
I20260811 17:14:32.652079 31070 raft_consensus.cc:2454] T ef9665d091f3448883f309d3cb013c98 P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cba9eb13c9442a4a4bd398dcd24aa8b in term 0.
I20260811 17:14:32.652086 30741 raft_consensus.cc:2454] T ef9665d091f3448883f309d3cb013c98 P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cba9eb13c9442a4a4bd398dcd24aa8b in term 0.
I20260811 17:14:32.652451 30829 leader_election.cc:302] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cba9eb13c9442a4a4bd398dcd24aa8b, fb61ec037ba54c36a368da1db9daa2d9; no voters:
I20260811 17:14:32.652740 31220 raft_consensus.cc:2781] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:14:32.652814 31220 raft_consensus.cc:482] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:14:32.652868 31220 raft_consensus.cc:3037] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:32.653985 31220 raft_consensus.cc:504] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:32.654351 31220 leader_election.cc:288] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b [CANDIDATE]: Term 1 election: Requested vote from peers 57339eab846a4db1a9c7980db2605dfa (127.21.54.129:35617), fb61ec037ba54c36a368da1db9daa2d9 (127.21.54.131:45139)
I20260811 17:14:32.654776 31070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef9665d091f3448883f309d3cb013c98" candidate_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb61ec037ba54c36a368da1db9daa2d9"
I20260811 17:14:32.654807 30741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef9665d091f3448883f309d3cb013c98" candidate_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57339eab846a4db1a9c7980db2605dfa"
I20260811 17:14:32.654920 31070 raft_consensus.cc:3037] T ef9665d091f3448883f309d3cb013c98 P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:32.654942 30741 raft_consensus.cc:3037] T ef9665d091f3448883f309d3cb013c98 P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:32.655850 31070 raft_consensus.cc:2454] T ef9665d091f3448883f309d3cb013c98 P fb61ec037ba54c36a368da1db9daa2d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cba9eb13c9442a4a4bd398dcd24aa8b in term 1.
I20260811 17:14:32.655915 30741 raft_consensus.cc:2454] T ef9665d091f3448883f309d3cb013c98 P 57339eab846a4db1a9c7980db2605dfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cba9eb13c9442a4a4bd398dcd24aa8b in term 1.
I20260811 17:14:32.656206 30829 leader_election.cc:302] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cba9eb13c9442a4a4bd398dcd24aa8b, fb61ec037ba54c36a368da1db9daa2d9; no voters:
I20260811 17:14:32.656376 31220 raft_consensus.cc:2781] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:32.656601 31220 raft_consensus.cc:686] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 1 LEADER]: Becoming Leader. State: Replica: 2cba9eb13c9442a4a4bd398dcd24aa8b, State: Running, Role: LEADER
I20260811 17:14:32.656790 31220 consensus_queue.cc:237] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:32.657485 31222 tablet_replica.cc:439] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b: TxnStatusTablet state changed. Reason: New leader 2cba9eb13c9442a4a4bd398dcd24aa8b. Latest consensus state: current_term: 1 leader_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } } }
I20260811 17:14:32.657604 31222 tablet_replica.cc:442] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b: This TxnStatusTablet replica's current role is: LEADER
I20260811 17:14:32.657902 31225 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 17:14:32.658097 31225 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 17:14:32.658686 30549 catalog_manager.cc:5495] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cba9eb13c9442a4a4bd398dcd24aa8b (127.21.54.130). New cstate: current_term: 1 leader_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } health_report { overall_health: UNKNOWN } } }
I20260811 17:14:32.692440 31174 consensus_queue.cc:1035] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:32.698486 31174 consensus_queue.cc:1035] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:32.703569 31200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:32.699080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37054 (local address 127.21.54.130:36755)
0811 17:14:32.699193 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:32.699211 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:32.699225 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:14:32.699375 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:32.699378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:32.699433 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:32.699504 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:14:32.700019 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.700925 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.702321 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.702682 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.702787 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:32.703098 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:14:32.703186 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:14:32.703266 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:32.703356 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 17:14:32.703419 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 17:14:32.703598 31211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:32.698748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:36755 (local address 127.0.0.1:37054)
0811 17:14:32.699105 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:32.699125 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:32.699237 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:32.699740 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:32.699743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:32.699752 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:14:32.699873 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:32.699880 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.701084 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.701088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:32.702162 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:32.702176 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.702815 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.702818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:32.702901 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:32.702951 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:14:32.703278 (+ 327us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:14:32.703289 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 17:14:32.703343 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 17:14:32.703420 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229}
I20260811 17:14:32.706282 31070 raft_consensus.cc:1261] T ef9665d091f3448883f309d3cb013c98 P fb61ec037ba54c36a368da1db9daa2d9 [term 1 FOLLOWER]: Refusing update from remote peer 2cba9eb13c9442a4a4bd398dcd24aa8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:32.706281 30741 raft_consensus.cc:1261] T ef9665d091f3448883f309d3cb013c98 P 57339eab846a4db1a9c7980db2605dfa [term 1 FOLLOWER]: Refusing update from remote peer 2cba9eb13c9442a4a4bd398dcd24aa8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:32.706712 31222 consensus_queue.cc:1035] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:32.706852 31220 consensus_queue.cc:1035] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:32.709033 31221 tablet_replica.cc:439] T ef9665d091f3448883f309d3cb013c98 P 57339eab846a4db1a9c7980db2605dfa: TxnStatusTablet state changed. Reason: New leader 2cba9eb13c9442a4a4bd398dcd24aa8b. Latest consensus state: current_term: 1 leader_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } } }
I20260811 17:14:32.709172 31221 tablet_replica.cc:442] T ef9665d091f3448883f309d3cb013c98 P 57339eab846a4db1a9c7980db2605dfa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:14:32.710141 31174 tablet_replica.cc:439] T ef9665d091f3448883f309d3cb013c98 P fb61ec037ba54c36a368da1db9daa2d9: TxnStatusTablet state changed. Reason: New leader 2cba9eb13c9442a4a4bd398dcd24aa8b. Latest consensus state: current_term: 1 leader_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } } }
I20260811 17:14:32.710268 31174 tablet_replica.cc:442] T ef9665d091f3448883f309d3cb013c98 P fb61ec037ba54c36a368da1db9daa2d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:14:32.710645 31220 tablet_replica.cc:439] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } } }
I20260811 17:14:32.710759 31220 tablet_replica.cc:442] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b: This TxnStatusTablet replica's current role is: LEADER
I20260811 17:14:32.710803 31222 tablet_replica.cc:439] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } } }
I20260811 17:14:32.710913 31222 tablet_replica.cc:442] T ef9665d091f3448883f309d3cb013c98 P 2cba9eb13c9442a4a4bd398dcd24aa8b: This TxnStatusTablet replica's current role is: LEADER
I20260811 17:14:32.712375 31174 tablet_replica.cc:439] T ef9665d091f3448883f309d3cb013c98 P fb61ec037ba54c36a368da1db9daa2d9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } } }
I20260811 17:14:32.712484 31174 tablet_replica.cc:442] T ef9665d091f3448883f309d3cb013c98 P fb61ec037ba54c36a368da1db9daa2d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:14:32.715317 31221 tablet_replica.cc:439] T ef9665d091f3448883f309d3cb013c98 P 57339eab846a4db1a9c7980db2605dfa: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } } }
I20260811 17:14:32.715445 31221 tablet_replica.cc:442] T ef9665d091f3448883f309d3cb013c98 P 57339eab846a4db1a9c7980db2605dfa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:14:32.722996 31247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:32.718035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48788 (local address 127.21.54.131:45139)
0811 17:14:32.718340 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:32.718350 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:32.718366 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:14:32.718455 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:32.718458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:32.718527 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:32.718669 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:14:32.718960 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.719886 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.720904 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.721309 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.721427 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:32.721762 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:14:32.722438 (+ 676us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:14:32.722564 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:32.722747 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 17:14:32.722812 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":114,"threads_started":1}
I20260811 17:14:32.735004 31250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:32.730874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:36755 (local address 127.21.54.131:55531)
0811 17:14:32.731197 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:32.731211 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:32.731308 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:32.731643 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:32.731646 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:32.731653 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:14:32.731764 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:32.731768 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.732918 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.732921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:32.733532 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:32.733540 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.734227 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.734230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:32.734352 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:32.734405 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:14:32.734733 (+ 328us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:14:32.734738 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 17:14:32.734792 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 17:14:32.734862 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":122,"threads_started":1}
I20260811 17:14:32.735054 31200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:32.730994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:55531 (local address 127.21.54.130:36755)
0811 17:14:32.731152 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:32.731157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:32.731285 (+ 128us) server_negotiation.cc:407] Connection header received
0811 17:14:32.731411 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:32.731414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:32.731477 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:32.731574 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:14:32.731879 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.732801 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.733689 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.734078 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.734190 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:32.734545 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:14:32.734641 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:14:32.734723 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:32.734833 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 17:14:32.734896 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 17:14:32.764634 31247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:32.760670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:47037 (local address 127.21.54.131:45139)
0811 17:14:32.760810 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:32.760815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:32.760830 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:14:32.760878 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:32.760881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:32.760931 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:32.761011 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:14:32.761308 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.762048 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.763158 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.763478 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.763609 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:32.764042 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:14:32.764134 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:14:32.764217 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:32.764382 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 17:14:32.764446 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 17:14:32.764925 31253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:32.760227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:45139 (local address 127.21.54.130:47037)
0811 17:14:32.760621 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:32.760638 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:32.760746 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:32.761067 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:32.761069 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:32.761076 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:14:32.761202 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:32.761206 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.762177 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.762181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:32.763020 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:32.763032 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.763705 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.763710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:32.763805 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:32.763875 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:14:32.764270 (+ 395us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:14:32.764278 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 17:14:32.764349 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 17:14:32.764428 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":149,"threads_started":1}
I20260811 17:14:32.797200 30552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:32.797399 30552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:32.800222 30552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc: Sending DeleteTablet for 3 replicas of tablet 86d05ec3331148b3b3f0ddc83dcf33a1
I20260811 17:14:32.800959 30885 tablet_service.cc:1514] Processing DeleteTablet for tablet 86d05ec3331148b3b3f0ddc83dcf33a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:32 UTC) from {username='oryx', principal='oryx/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:37038
I20260811 17:14:32.801385 30718 tablet_service.cc:1514] Processing DeleteTablet for tablet 86d05ec3331148b3b3f0ddc83dcf33a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:32 UTC) from {username='oryx', principal='oryx/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:38214
I20260811 17:14:32.801421 31054 tablet_service.cc:1514] Processing DeleteTablet for tablet 86d05ec3331148b3b3f0ddc83dcf33a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:32 UTC) from {username='oryx', principal='oryx/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:48784
I20260811 17:14:32.801941 31260 tablet_replica.cc:330] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9: stopping tablet replica
I20260811 17:14:32.802131 31260 raft_consensus.cc:2229] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:14:32.802596 31260 raft_consensus.cc:2258] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:32.803450 31260 ts_tablet_manager.cc:1905] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:32.804349 31257 tablet_replica.cc:330] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b: stopping tablet replica
I20260811 17:14:32.804587 31257 raft_consensus.cc:2229] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:32.804780 31257 raft_consensus.cc:2258] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:32.806026 31257 ts_tablet_manager.cc:1905] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:32.808791 31258 tablet_replica.cc:330] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa: stopping tablet replica
I20260811 17:14:32.809007 31258 raft_consensus.cc:2229] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:32.809175 31258 raft_consensus.cc:2258] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:32.809890 31258 ts_tablet_manager.cc:1905] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:32.811537 31257 ts_tablet_manager.cc:1918] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:14:32.812153 31260 ts_tablet_manager.cc:1918] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:14:32.812290 31260 log.cc:1196] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/86d05ec3331148b3b3f0ddc83dcf33a1
I20260811 17:14:32.812665 31260 ts_tablet_manager.cc:1939] T 86d05ec3331148b3b3f0ddc83dcf33a1 P fb61ec037ba54c36a368da1db9daa2d9: Deleting consensus metadata
I20260811 17:14:32.813308 31258 ts_tablet_manager.cc:1918] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:14:32.813395 31258 log.cc:1196] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/wal/wals/86d05ec3331148b3b3f0ddc83dcf33a1
I20260811 17:14:32.813586 30537 catalog_manager.cc:4841] TS fb61ec037ba54c36a368da1db9daa2d9 (127.21.54.131:45139): tablet 86d05ec3331148b3b3f0ddc83dcf33a1 (table test-table [id=2588764161f0482cabaa93aced172a84]) successfully deleted
I20260811 17:14:32.813653 31258 ts_tablet_manager.cc:1939] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 57339eab846a4db1a9c7980db2605dfa: Deleting consensus metadata
I20260811 17:14:32.813897 31257 log.cc:1196] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/86d05ec3331148b3b3f0ddc83dcf33a1
I20260811 17:14:32.814265 31257 ts_tablet_manager.cc:1939] T 86d05ec3331148b3b3f0ddc83dcf33a1 P 2cba9eb13c9442a4a4bd398dcd24aa8b: Deleting consensus metadata
I20260811 17:14:32.814421 30534 catalog_manager.cc:4841] TS 57339eab846a4db1a9c7980db2605dfa (127.21.54.129:35617): tablet 86d05ec3331148b3b3f0ddc83dcf33a1 (table test-table [id=2588764161f0482cabaa93aced172a84]) successfully deleted
I20260811 17:14:32.815173 30535 catalog_manager.cc:4841] TS 2cba9eb13c9442a4a4bd398dcd24aa8b (127.21.54.130:36755): tablet 86d05ec3331148b3b3f0ddc83dcf33a1 (table test-table [id=2588764161f0482cabaa93aced172a84]) successfully deleted
Aug 11 17:14:32 dist-test-slave-fd7l krb5kdc[30498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 17:14:32 dist-test-slave-fd7l krb5kdc[30498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:32 dist-test-slave-fd7l krb5kdc[30498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 17:14:32.846511 31212 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:32.838599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51208 (local address 127.21.54.190:42881)
0811 17:14:32.838778 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:32.838784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:32.838975 (+ 191us) server_negotiation.cc:407] Connection header received
0811 17:14:32.839138 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:32.839142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:32.839198 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:32.839299 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:32.840233 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.841240 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.842413 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.842804 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.842906 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:32.846347 (+ 3441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51208: BlockingRecv error: recv got EOF from 127.0.0.1:51208 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 17:14:32 dist-test-slave-fd7l krb5kdc[30498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM
I20260811 17:14:32.860368 31212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:32.848115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51216 (local address 127.21.54.190:42881)
0811 17:14:32.848247 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:32.848251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:32.848810 (+ 559us) server_negotiation.cc:407] Connection header received
0811 17:14:32.849004 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:32.849007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:32.849070 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:32.849123 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:32.850047 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.851035 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.852189 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:32.852405 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:32.852487 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:32.855580 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:32.855585 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:32.855588 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:32.855623 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:32.857510 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:32.858038 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:32.858042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:32.858044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:32.858150 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:32.858500 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:32.858548 (+ 48us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:32.858552 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:32.859118 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:32.859441 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:32.859729 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 17:14:32.860047 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 17:14:32.864203 30552 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51216:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:14:32.864787 30552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:14:32.870682 30718 tablet_service.cc:1467] Processing CreateTablet for tablet 475eb8504e924d639c1a468c1c13339d (DEFAULT_TABLE table=test-table [id=d4c53e611a22491e9be296a433819864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:32.870801 30885 tablet_service.cc:1467] Processing CreateTablet for tablet 475eb8504e924d639c1a468c1c13339d (DEFAULT_TABLE table=test-table [id=d4c53e611a22491e9be296a433819864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:32.871090 30718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 475eb8504e924d639c1a468c1c13339d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:32.871140 30885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 475eb8504e924d639c1a468c1c13339d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:32.874006 31054 tablet_service.cc:1467] Processing CreateTablet for tablet 475eb8504e924d639c1a468c1c13339d (DEFAULT_TABLE table=test-table [id=d4c53e611a22491e9be296a433819864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:32.874351 31054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 475eb8504e924d639c1a468c1c13339d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:32.876617 31213 tablet_bootstrap.cc:492] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b: Bootstrap starting.
I20260811 17:14:32.877518 31213 tablet_bootstrap.cc:654] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:32.878383 31216 tablet_bootstrap.cc:492] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa: Bootstrap starting.
I20260811 17:14:32.878998 31213 tablet_bootstrap.cc:492] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b: No bootstrap required, opened a new log
I20260811 17:14:32.879110 31213 ts_tablet_manager.cc:1397] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:14:32.879359 31216 tablet_bootstrap.cc:654] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:32.879653 31213 raft_consensus.cc:348] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:32.879801 31213 raft_consensus.cc:374] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:32.879848 31213 raft_consensus.cc:729] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cba9eb13c9442a4a4bd398dcd24aa8b, State: Initialized, Role: FOLLOWER
I20260811 17:14:32.880008 31213 consensus_queue.cc:260] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:32.880080 31215 tablet_bootstrap.cc:492] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9: Bootstrap starting.
I20260811 17:14:32.880482 31213 ts_tablet_manager.cc:1428] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 17:14:32.881114 31215 tablet_bootstrap.cc:654] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:32.882572 31215 tablet_bootstrap.cc:492] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9: No bootstrap required, opened a new log
I20260811 17:14:32.882692 31215 ts_tablet_manager.cc:1397] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 17:14:32.883101 31215 raft_consensus.cc:348] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:32.883201 31215 raft_consensus.cc:374] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:32.883224 31215 raft_consensus.cc:729] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb61ec037ba54c36a368da1db9daa2d9, State: Initialized, Role: FOLLOWER
I20260811 17:14:32.883384 31215 consensus_queue.cc:260] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:32.883697 31215 ts_tablet_manager.cc:1428] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 17:14:32.884827 31216 tablet_bootstrap.cc:492] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa: No bootstrap required, opened a new log
I20260811 17:14:32.884951 31216 ts_tablet_manager.cc:1397] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 17:14:32.885502 31216 raft_consensus.cc:348] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:32.885648 31216 raft_consensus.cc:374] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:32.885691 31216 raft_consensus.cc:729] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57339eab846a4db1a9c7980db2605dfa, State: Initialized, Role: FOLLOWER
I20260811 17:14:32.885843 31216 consensus_queue.cc:260] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:32.886180 31216 ts_tablet_manager.cc:1428] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:14:33.056422 31174 raft_consensus.cc:482] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:14:33.056699 31174 raft_consensus.cc:504] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:33.057201 31174 leader_election.cc:288] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57339eab846a4db1a9c7980db2605dfa (127.21.54.129:35617), 2cba9eb13c9442a4a4bd398dcd24aa8b (127.21.54.130:36755)
I20260811 17:14:33.057756 30910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "475eb8504e924d639c1a468c1c13339d" candidate_uuid: "fb61ec037ba54c36a368da1db9daa2d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" is_pre_election: true
I20260811 17:14:33.057783 30741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "475eb8504e924d639c1a468c1c13339d" candidate_uuid: "fb61ec037ba54c36a368da1db9daa2d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57339eab846a4db1a9c7980db2605dfa" is_pre_election: true
I20260811 17:14:33.057967 30741 raft_consensus.cc:2454] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb61ec037ba54c36a368da1db9daa2d9 in term 0.
I20260811 17:14:33.057948 30910 raft_consensus.cc:2454] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb61ec037ba54c36a368da1db9daa2d9 in term 0.
I20260811 17:14:33.058341 30993 leader_election.cc:302] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57339eab846a4db1a9c7980db2605dfa, fb61ec037ba54c36a368da1db9daa2d9; no voters:
I20260811 17:14:33.058562 31174 raft_consensus.cc:2781] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:14:33.058626 31174 raft_consensus.cc:482] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:14:33.058656 31174 raft_consensus.cc:3037] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:33.059808 31174 raft_consensus.cc:504] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:33.060263 31174 leader_election.cc:288] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [CANDIDATE]: Term 1 election: Requested vote from peers 57339eab846a4db1a9c7980db2605dfa (127.21.54.129:35617), 2cba9eb13c9442a4a4bd398dcd24aa8b (127.21.54.130:36755)
I20260811 17:14:33.060701 30741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "475eb8504e924d639c1a468c1c13339d" candidate_uuid: "fb61ec037ba54c36a368da1db9daa2d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57339eab846a4db1a9c7980db2605dfa"
I20260811 17:14:33.060882 30741 raft_consensus.cc:3037] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:33.060878 30910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "475eb8504e924d639c1a468c1c13339d" candidate_uuid: "fb61ec037ba54c36a368da1db9daa2d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b"
I20260811 17:14:33.061054 30910 raft_consensus.cc:3037] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:33.061996 30741 raft_consensus.cc:2454] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb61ec037ba54c36a368da1db9daa2d9 in term 1.
I20260811 17:14:33.062093 30910 raft_consensus.cc:2454] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb61ec037ba54c36a368da1db9daa2d9 in term 1.
I20260811 17:14:33.062391 30993 leader_election.cc:302] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57339eab846a4db1a9c7980db2605dfa, fb61ec037ba54c36a368da1db9daa2d9; no voters:
I20260811 17:14:33.062604 31174 raft_consensus.cc:2781] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:33.062705 31174 raft_consensus.cc:686] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [term 1 LEADER]: Becoming Leader. State: Replica: fb61ec037ba54c36a368da1db9daa2d9, State: Running, Role: LEADER
I20260811 17:14:33.062865 31174 consensus_queue.cc:237] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:33.064985 30548 catalog_manager.cc:5495] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb61ec037ba54c36a368da1db9daa2d9 (127.21.54.131). New cstate: current_term: 1 leader_uuid: "fb61ec037ba54c36a368da1db9daa2d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } health_report { overall_health: HEALTHY } } }
I20260811 17:14:33.120225 31247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:33.116542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48790 (local address 127.21.54.131:45139)
0811 17:14:33.116707 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:33.116711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:33.116729 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:14:33.116911 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:33.116914 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:33.116967 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:33.117080 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:14:33.117378 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.118005 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.118970 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.119221 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.119354 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:33.119669 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:14:33.119755 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:14:33.119847 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:33.119987 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 17:14:33.120050 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 17:14:33.122902 30741 raft_consensus.cc:1261] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa [term 1 FOLLOWER]: Refusing update from remote peer fb61ec037ba54c36a368da1db9daa2d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:33.122942 30910 raft_consensus.cc:1261] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 1 FOLLOWER]: Refusing update from remote peer fb61ec037ba54c36a368da1db9daa2d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:33.123468 31251 consensus_queue.cc:1035] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:33.123695 31174 consensus_queue.cc:1035] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:33.133366 30548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:33.133519 30548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:33.135645 30548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc: Sending DeleteTablet for 3 replicas of tablet 475eb8504e924d639c1a468c1c13339d
I20260811 17:14:33.136348 30718 tablet_service.cc:1514] Processing DeleteTablet for tablet 475eb8504e924d639c1a468c1c13339d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:33 UTC) from {username='oryx', principal='oryx/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:38214
I20260811 17:14:33.136399 31054 tablet_service.cc:1514] Processing DeleteTablet for tablet 475eb8504e924d639c1a468c1c13339d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:33 UTC) from {username='oryx', principal='oryx/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:48784
I20260811 17:14:33.136617 31258 tablet_replica.cc:330] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa: stopping tablet replica
I20260811 17:14:33.136649 30885 tablet_service.cc:1514] Processing DeleteTablet for tablet 475eb8504e924d639c1a468c1c13339d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:33 UTC) from {username='oryx', principal='oryx/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:37038
I20260811 17:14:33.136766 31258 raft_consensus.cc:2229] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:33.136876 31257 tablet_replica.cc:330] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b: stopping tablet replica
I20260811 17:14:33.136883 31260 tablet_replica.cc:330] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9: stopping tablet replica
I20260811 17:14:33.136927 31258 raft_consensus.cc:2258] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:33.137001 31257 raft_consensus.cc:2229] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:33.137068 31260 raft_consensus.cc:2229] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:14:33.137133 31257 raft_consensus.cc:2258] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:33.137302 31260 raft_consensus.cc:2258] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:33.137918 30548 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51216:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 17:14:33.138244 31257 ts_tablet_manager.cc:1905] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:33.138302 31258 ts_tablet_manager.cc:1905] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:33.138425 31260 ts_tablet_manager.cc:1905] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 17:14:33.138751 30548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:14:33.140936 31258 ts_tablet_manager.cc:1918] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:14:33.141029 31258 log.cc:1196] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/wal/wals/475eb8504e924d639c1a468c1c13339d
I20260811 17:14:33.141146 31257 ts_tablet_manager.cc:1918] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:14:33.141147 31260 ts_tablet_manager.cc:1918] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:14:33.141232 31260 log.cc:1196] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/475eb8504e924d639c1a468c1c13339d
I20260811 17:14:33.141232 31257 log.cc:1196] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/475eb8504e924d639c1a468c1c13339d
I20260811 17:14:33.141366 31258 ts_tablet_manager.cc:1939] T 475eb8504e924d639c1a468c1c13339d P 57339eab846a4db1a9c7980db2605dfa: Deleting consensus metadata
I20260811 17:14:33.141496 31257 ts_tablet_manager.cc:1939] T 475eb8504e924d639c1a468c1c13339d P 2cba9eb13c9442a4a4bd398dcd24aa8b: Deleting consensus metadata
I20260811 17:14:33.141606 31260 ts_tablet_manager.cc:1939] T 475eb8504e924d639c1a468c1c13339d P fb61ec037ba54c36a368da1db9daa2d9: Deleting consensus metadata
I20260811 17:14:33.142241 30534 catalog_manager.cc:4841] TS 57339eab846a4db1a9c7980db2605dfa (127.21.54.129:35617): tablet 475eb8504e924d639c1a468c1c13339d (table test-table [id=d4c53e611a22491e9be296a433819864]) successfully deleted
I20260811 17:14:33.142621 30537 catalog_manager.cc:4841] TS fb61ec037ba54c36a368da1db9daa2d9 (127.21.54.131:45139): tablet 475eb8504e924d639c1a468c1c13339d (table test-table [id=d4c53e611a22491e9be296a433819864]) successfully deleted
I20260811 17:14:33.143231 30535 catalog_manager.cc:4841] TS 2cba9eb13c9442a4a4bd398dcd24aa8b (127.21.54.130:36755): tablet 475eb8504e924d639c1a468c1c13339d (table test-table [id=d4c53e611a22491e9be296a433819864]) successfully deleted
I20260811 17:14:33.145093 30718 tablet_service.cc:1467] Processing CreateTablet for tablet a3b4521b0cc54133b289f8dd3299f3bd (DEFAULT_TABLE table=test-table [id=1768d79a216148f99e1dd514cb4ebdfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:33.145427 30718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3b4521b0cc54133b289f8dd3299f3bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:33.145433 31054 tablet_service.cc:1467] Processing CreateTablet for tablet a3b4521b0cc54133b289f8dd3299f3bd (DEFAULT_TABLE table=test-table [id=1768d79a216148f99e1dd514cb4ebdfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:33.145749 31054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3b4521b0cc54133b289f8dd3299f3bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:33.147696 31216 tablet_bootstrap.cc:492] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa: Bootstrap starting.
I20260811 17:14:33.148048 31215 tablet_bootstrap.cc:492] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9: Bootstrap starting.
I20260811 17:14:33.148281 30885 tablet_service.cc:1467] Processing CreateTablet for tablet a3b4521b0cc54133b289f8dd3299f3bd (DEFAULT_TABLE table=test-table [id=1768d79a216148f99e1dd514cb4ebdfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:33.148615 30885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3b4521b0cc54133b289f8dd3299f3bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:33.148797 31216 tablet_bootstrap.cc:654] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:33.149024 31215 tablet_bootstrap.cc:654] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:33.150107 31215 tablet_bootstrap.cc:492] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9: No bootstrap required, opened a new log
I20260811 17:14:33.150229 31215 ts_tablet_manager.cc:1397] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 17:14:33.150609 31215 raft_consensus.cc:348] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:33.150730 31215 raft_consensus.cc:374] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:33.150775 31215 raft_consensus.cc:729] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb61ec037ba54c36a368da1db9daa2d9, State: Initialized, Role: FOLLOWER
I20260811 17:14:33.150919 31215 consensus_queue.cc:260] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:33.151068 31213 tablet_bootstrap.cc:492] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b: Bootstrap starting.
I20260811 17:14:33.151199 31215 ts_tablet_manager.cc:1428] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:14:33.152021 31213 tablet_bootstrap.cc:654] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:33.153249 31213 tablet_bootstrap.cc:492] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b: No bootstrap required, opened a new log
I20260811 17:14:33.153347 31213 ts_tablet_manager.cc:1397] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 17:14:33.153429 31216 tablet_bootstrap.cc:492] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa: No bootstrap required, opened a new log
I20260811 17:14:33.153584 31216 ts_tablet_manager.cc:1397] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 17:14:33.153782 31213 raft_consensus.cc:348] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:33.153926 31213 raft_consensus.cc:374] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:33.153961 31213 raft_consensus.cc:729] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cba9eb13c9442a4a4bd398dcd24aa8b, State: Initialized, Role: FOLLOWER
I20260811 17:14:33.154116 31213 consensus_queue.cc:260] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:33.154115 31216 raft_consensus.cc:348] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:33.154263 31216 raft_consensus.cc:374] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:33.154311 31216 raft_consensus.cc:729] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57339eab846a4db1a9c7980db2605dfa, State: Initialized, Role: FOLLOWER
I20260811 17:14:33.154415 31213 ts_tablet_manager.cc:1428] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 17:14:33.154625 31216 consensus_queue.cc:260] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:33.155035 31216 ts_tablet_manager.cc:1428] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 17:14:33.348538 31221 raft_consensus.cc:482] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:14:33.348709 31221 raft_consensus.cc:504] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:33.349911 31221 leader_election.cc:288] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cba9eb13c9442a4a4bd398dcd24aa8b (127.21.54.130:36755), fb61ec037ba54c36a368da1db9daa2d9 (127.21.54.131:45139)
I20260811 17:14:33.355759 31247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:33.350325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:57775 (local address 127.21.54.131:45139)
0811 17:14:33.350472 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:33.350478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:33.350802 (+ 324us) server_negotiation.cc:407] Connection header received
0811 17:14:33.350902 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:33.350908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:33.350964 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:33.351071 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:33.351408 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.352218 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.354245 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.354873 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.355010 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:33.355069 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:33.355529 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 17:14:33.355622 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 17:14:33.355832 31287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:33.350222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:36755 (local address 127.21.54.129:52211)
0811 17:14:33.350631 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:33.350646 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:33.350811 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:33.351100 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:33.351103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:33.351176 (+ 73us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:33.351293 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:33.351298 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.352367 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.352373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:33.354075 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:33.354088 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.355011 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.355015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:33.355294 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:33.355404 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 17:14:33.355487 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 17:14:33.355619 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":192,"spinlock_wait_cycles":13440,"thread_start_us":82,"threads_started":1}
I20260811 17:14:33.355794 31288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:33.350481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:52211 (local address 127.21.54.130:36755)
0811 17:14:33.350787 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:33.350792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:33.350809 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:14:33.350936 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:33.350939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:33.350984 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:33.351073 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:33.351413 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.352230 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.354245 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.354875 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.355010 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:33.355069 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:33.355529 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 17:14:33.355670 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":73,"threads_started":1}
I20260811 17:14:33.355831 31286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:33.350222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:45139 (local address 127.21.54.129:57775)
0811 17:14:33.350707 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:33.350721 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:33.350799 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:33.351100 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:33.351103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:33.351130 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:33.351278 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:33.351286 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.352381 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.352384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:33.354075 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:33.354088 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.355011 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.355015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:33.355293 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:33.355406 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 17:14:33.355487 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 17:14:33.355619 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":450,"spinlock_wait_cycles":1408,"thread_start_us":89,"threads_started":1}
I20260811 17:14:33.356470 30910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3b4521b0cc54133b289f8dd3299f3bd" candidate_uuid: "57339eab846a4db1a9c7980db2605dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" is_pre_election: true
I20260811 17:14:33.356632 31070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3b4521b0cc54133b289f8dd3299f3bd" candidate_uuid: "57339eab846a4db1a9c7980db2605dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb61ec037ba54c36a368da1db9daa2d9" is_pre_election: true
I20260811 17:14:33.356689 30910 raft_consensus.cc:2454] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57339eab846a4db1a9c7980db2605dfa in term 0.
I20260811 17:14:33.356779 31070 raft_consensus.cc:2454] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57339eab846a4db1a9c7980db2605dfa in term 0.
I20260811 17:14:33.357177 30657 leader_election.cc:302] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2cba9eb13c9442a4a4bd398dcd24aa8b, 57339eab846a4db1a9c7980db2605dfa, fb61ec037ba54c36a368da1db9daa2d9; no voters:
I20260811 17:14:33.357503 31221 raft_consensus.cc:2781] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:14:33.357565 31221 raft_consensus.cc:482] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:14:33.357597 31221 raft_consensus.cc:3037] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:33.358754 31221 raft_consensus.cc:504] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:33.359135 31221 leader_election.cc:288] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [CANDIDATE]: Term 1 election: Requested vote from peers 2cba9eb13c9442a4a4bd398dcd24aa8b (127.21.54.130:36755), fb61ec037ba54c36a368da1db9daa2d9 (127.21.54.131:45139)
I20260811 17:14:33.359519 30910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3b4521b0cc54133b289f8dd3299f3bd" candidate_uuid: "57339eab846a4db1a9c7980db2605dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b"
I20260811 17:14:33.359550 31070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3b4521b0cc54133b289f8dd3299f3bd" candidate_uuid: "57339eab846a4db1a9c7980db2605dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb61ec037ba54c36a368da1db9daa2d9"
I20260811 17:14:33.359665 31070 raft_consensus.cc:3037] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:33.359665 30910 raft_consensus.cc:3037] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:33.360885 30910 raft_consensus.cc:2454] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57339eab846a4db1a9c7980db2605dfa in term 1.
I20260811 17:14:33.360885 31070 raft_consensus.cc:2454] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57339eab846a4db1a9c7980db2605dfa in term 1.
I20260811 17:14:33.361264 30659 leader_election.cc:302] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57339eab846a4db1a9c7980db2605dfa, fb61ec037ba54c36a368da1db9daa2d9; no voters:
I20260811 17:14:33.361457 31221 raft_consensus.cc:2781] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:33.361683 31221 raft_consensus.cc:686] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [term 1 LEADER]: Becoming Leader. State: Replica: 57339eab846a4db1a9c7980db2605dfa, State: Running, Role: LEADER
I20260811 17:14:33.361953 31221 consensus_queue.cc:237] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } }
I20260811 17:14:33.364377 30548 catalog_manager.cc:5495] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa reported cstate change: term changed from 0 to 1, leader changed from <none> to 57339eab846a4db1a9c7980db2605dfa (127.21.54.129). New cstate: current_term: 1 leader_uuid: "57339eab846a4db1a9c7980db2605dfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57339eab846a4db1a9c7980db2605dfa" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 } health_report { overall_health: UNKNOWN } } }
I20260811 17:14:33.399848 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:33.395189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50050 (local address 127.21.54.129:35617)
0811 17:14:33.395609 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:33.395617 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:33.395641 (+ 24us) server_negotiation.cc:407] Connection header received
0811 17:14:33.395730 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:33.395732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:33.395790 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:33.395908 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:14:33.396243 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.396895 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.397847 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.398101 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.398190 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:33.398535 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:14:33.399150 (+ 615us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:14:33.399258 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:33.399442 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 17:14:33.399609 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":159,"threads_started":1}
I20260811 17:14:33.412143 31299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:33.408254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:36755 (local address 127.21.54.129:34479)
0811 17:14:33.408720 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:33.408733 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:33.408865 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:33.409158 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:33.409162 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:33.409169 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:14:33.409300 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:33.409311 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.410156 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.410161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:33.410839 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:33.410848 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.411336 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.411340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:33.411442 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:33.411504 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:14:33.411828 (+ 324us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:14:33.411837 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 17:14:33.411904 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 17:14:33.411989 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":146,"threads_started":1}
I20260811 17:14:33.412129 31288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:33.408415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:34479 (local address 127.21.54.130:36755)
0811 17:14:33.408638 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:33.408643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:33.408811 (+ 168us) server_negotiation.cc:407] Connection header received
0811 17:14:33.409006 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:33.409009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:33.409056 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:33.409146 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:14:33.409416 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.410037 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.410990 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.411210 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.411334 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:33.411661 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:14:33.411737 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:14:33.411820 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:33.411914 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 17:14:33.411977 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 17:14:33.418541 31070 raft_consensus.cc:1261] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9 [term 1 FOLLOWER]: Refusing update from remote peer 57339eab846a4db1a9c7980db2605dfa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:33.418561 30910 raft_consensus.cc:1261] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 1 FOLLOWER]: Refusing update from remote peer 57339eab846a4db1a9c7980db2605dfa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:33.419339 31289 consensus_queue.cc:1035] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb61ec037ba54c36a368da1db9daa2d9" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 45139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:33.419533 31221 consensus_queue.cc:1035] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cba9eb13c9442a4a4bd398dcd24aa8b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:33.440203 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:33.435983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:52327 (local address 127.21.54.129:35617)
0811 17:14:33.436257 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:33.436262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:33.436422 (+ 160us) server_negotiation.cc:407] Connection header received
0811 17:14:33.436600 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:33.436603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:33.436664 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:33.436750 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:14:33.437002 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.437601 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.438967 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.439302 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.439416 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:33.439718 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:14:33.439823 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:14:33.439904 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:33.439985 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 17:14:33.440084 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260811 17:14:33.440243 31307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:33.435909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:35617 (local address 127.21.54.130:52327)
0811 17:14:33.436316 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:33.436331 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:33.436471 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:33.436759 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:33.436762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:33.436771 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:14:33.436878 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:33.436884 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.437782 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.437785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:33.438821 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:33.438831 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:33.439438 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:33.439445 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:33.439542 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:33.439579 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:14:33.439915 (+ 336us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:14:33.439921 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 17:14:33.439977 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 17:14:33.440064 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":134,"threads_started":1}
I20260811 17:14:33.465842 30548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:33.466084 30548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:33.469954 30548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a4c1bf54035469295770754b7bf6fcc: Sending DeleteTablet for 3 replicas of tablet a3b4521b0cc54133b289f8dd3299f3bd
I20260811 17:14:33.470789 31054 tablet_service.cc:1514] Processing DeleteTablet for tablet a3b4521b0cc54133b289f8dd3299f3bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:33 UTC) from {username='oryx', principal='oryx/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:48784
I20260811 17:14:33.470973 30718 tablet_service.cc:1514] Processing DeleteTablet for tablet a3b4521b0cc54133b289f8dd3299f3bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:33 UTC) from {username='oryx', principal='oryx/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:38214
I20260811 17:14:33.471061 31260 tablet_replica.cc:330] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9: stopping tablet replica
I20260811 17:14:33.471208 31258 tablet_replica.cc:330] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa: stopping tablet replica
I20260811 17:14:33.471208 31260 raft_consensus.cc:2229] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:33.471330 31258 raft_consensus.cc:2229] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:14:33.471349 31260 raft_consensus.cc:2258] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:33.471547 30885 tablet_service.cc:1514] Processing DeleteTablet for tablet a3b4521b0cc54133b289f8dd3299f3bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:33 UTC) from {username='oryx', principal='oryx/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:37038
I20260811 17:14:33.471700 31258 raft_consensus.cc:2258] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:33.471889 31257 tablet_replica.cc:330] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b: stopping tablet replica
I20260811 17:14:33.472024 31257 raft_consensus.cc:2229] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:33.472144 31257 raft_consensus.cc:2258] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:33.472216 31260 ts_tablet_manager.cc:1905] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:33.472726 31258 ts_tablet_manager.cc:1905] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:33.473064 31257 ts_tablet_manager.cc:1905] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:33.475070 31260 ts_tablet_manager.cc:1918] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:14:33.475158 31260 log.cc:1196] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/a3b4521b0cc54133b289f8dd3299f3bd
I20260811 17:14:33.475426 31260 ts_tablet_manager.cc:1939] T a3b4521b0cc54133b289f8dd3299f3bd P fb61ec037ba54c36a368da1db9daa2d9: Deleting consensus metadata
I20260811 17:14:33.475803 31258 ts_tablet_manager.cc:1918] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:14:33.475890 31258 log.cc:1196] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-0/wal/wals/a3b4521b0cc54133b289f8dd3299f3bd
I20260811 17:14:33.476167 31258 ts_tablet_manager.cc:1939] T a3b4521b0cc54133b289f8dd3299f3bd P 57339eab846a4db1a9c7980db2605dfa: Deleting consensus metadata
I20260811 17:14:33.476295 31257 ts_tablet_manager.cc:1918] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:14:33.476423 31257 log.cc:1196] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/a3b4521b0cc54133b289f8dd3299f3bd
I20260811 17:14:33.476752 30537 catalog_manager.cc:4841] TS fb61ec037ba54c36a368da1db9daa2d9 (127.21.54.131:45139): tablet a3b4521b0cc54133b289f8dd3299f3bd (table test-table [id=1768d79a216148f99e1dd514cb4ebdfe]) successfully deleted
I20260811 17:14:33.476828 31257 ts_tablet_manager.cc:1939] T a3b4521b0cc54133b289f8dd3299f3bd P 2cba9eb13c9442a4a4bd398dcd24aa8b: Deleting consensus metadata
I20260811 17:14:33.477746 30535 catalog_manager.cc:4841] TS 2cba9eb13c9442a4a4bd398dcd24aa8b (127.21.54.130:36755): tablet a3b4521b0cc54133b289f8dd3299f3bd (table test-table [id=1768d79a216148f99e1dd514cb4ebdfe]) successfully deleted
I20260811 17:14:33.478269 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 30639
I20260811 17:14:33.489879 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 30809
W20260811 17:14:33.489912 30534 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.54.129:35617 (error 108)
I20260811 17:14:33.494716 31312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:33.494178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:35617 (local address 127.0.0.1:50058)
0811 17:14:33.494536 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:33.494611 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.129:35617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":69,"threads_started":1}
W20260811 17:14:33.494902 30534 catalog_manager.cc:4584] TS 57339eab846a4db1a9c7980db2605dfa (127.21.54.129:35617): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a3b4521b0cc54133b289f8dd3299f3bd: Network error: Client connection negotiation failed: client connection to 127.21.54.129:35617: connect: Connection refused (error 111)
I20260811 17:14:33.502257 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 30974
I20260811 17:14:33.514235 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 30512
2026-08-11T17:14:33Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (4236 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:14:33 dist-test-slave-fd7l krb5kdc[31315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:14:33 dist-test-slave-fd7l krb5kdc[31315](info): set up 2 sockets
Aug 11 17:14:33 dist-test-slave-fd7l krb5kdc[31315](info): commencing operation
krb5kdc: starting...
W20260811 17:14:34.658717 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:14:34 dist-test-slave-fd7l krb5kdc[31315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:14:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:14:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:14:34.804641 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:40405
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:40405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:34.926880 31331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:34.927248 31331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:34.927340 31331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:34.931547 31331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:34.931665 31331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:34.931702 31331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:34.931746 31331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:34.931782 31331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:14:34.937448 31331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:40405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:40405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:34.938692 31331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:34.939797 31331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:34.949198 31337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:34.949198 31336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:34.949221 31339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:34.949323 31331 server_base.cc:1034] running on GCE node
I20260811 17:14:34.950186 31331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:34.951469 31331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:34.952704 31331 hybrid_clock.cc:648] HybridClock initialized: now 1786468474952686 us; error 35 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T17:14:34Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1443 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:14:35 dist-test-slave-fd7l krb5kdc[31347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:14:35 dist-test-slave-fd7l krb5kdc[31347](info): set up 2 sockets
Aug 11 17:14:35 dist-test-slave-fd7l krb5kdc[31347](info): commencing operation
krb5kdc: starting...
W20260811 17:14:36.123481 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:14:36 dist-test-slave-fd7l krb5kdc[31347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:14:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:14:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:14:36.269706 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:35349
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:35349
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:36.390331 31362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:36.390630 31362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:36.390679 31362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:36.394829 31362 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:36.394912 31362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:36.394932 31362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:36.394949 31362 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:36.394964 31362 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:14:36.400862 31362 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:35349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:35349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:36.402104 31362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:36.403251 31362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:36.412284 31371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:36.412355 31362 server_base.cc:1034] running on GCE node
W20260811 17:14:36.412293 31369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:36.412293 31368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:36.413221 31362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:36.414402 31362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:36.415567 31362 hybrid_clock.cc:648] HybridClock initialized: now 1786468476415542 us; error 41 us; skew 500 ppm
Aug 11 17:14:36 dist-test-slave-fd7l krb5kdc[31347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:36.423183 31362 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:14:36.424181 31362 webserver.cc:460] Webserver started at http://127.21.54.190:33917/ using document root <none> and password file <none>
I20260811 17:14:36.424809 31362 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:36.424875 31362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:36.425050 31362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:36.427152 31362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1"
format_stamp: "Formatted at 2026-08-11 17:14:36 on dist-test-slave-fd7l"
server_key: "624957181d3b0f002ead9d14349b0c25"
server_key_iv: "a1ec28b8f213d7509cda7da7ebefaaba"
server_key_version: "encryptionkey@0"
I20260811 17:14:36.427713 31362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1"
format_stamp: "Formatted at 2026-08-11 17:14:36 on dist-test-slave-fd7l"
server_key: "624957181d3b0f002ead9d14349b0c25"
server_key_iv: "a1ec28b8f213d7509cda7da7ebefaaba"
server_key_version: "encryptionkey@0"
I20260811 17:14:36.431476 31362 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 17:14:36.433851 31378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:36.435104 31362 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 17:14:36.435250 31362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1"
format_stamp: "Formatted at 2026-08-11 17:14:36 on dist-test-slave-fd7l"
server_key: "624957181d3b0f002ead9d14349b0c25"
server_key_iv: "a1ec28b8f213d7509cda7da7ebefaaba"
server_key_version: "encryptionkey@0"
I20260811 17:14:36.435364 31362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:36.448181 31362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:36.451228 31362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:36.451440 31362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:36.458993 31362 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:35349
I20260811 17:14:36.459009 31443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:35349 every 8 connection(s)
I20260811 17:14:36.460000 31362 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:14:36.463008 31444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:36.468127 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 31362
I20260811 17:14:36.468276 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:14:36.468645 21722 external_mini_cluster.cc:1428] Setting key 48637d323711252a0487b73e1eb1260f
I20260811 17:14:36.468811 31444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1: Bootstrap starting.
I20260811 17:14:36.471347 31444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:36.472313 31444 log.cc:824] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:36.474610 31444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1: No bootstrap required, opened a new log
Aug 11 17:14:36 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:36.479475 31444 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35349 } }
I20260811 17:14:36.479745 31444 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:36.479828 31444 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c1ceb0e5384e91a73712c7f52fc0b1, State: Initialized, Role: FOLLOWER
I20260811 17:14:36.480353 31444 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35349 } }
I20260811 17:14:36.480517 31444 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:14:36.480618 31444 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:14:36.480731 31444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:36.481717 31444 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35349 } }
I20260811 17:14:36.482103 31444 leader_election.cc:302] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8c1ceb0e5384e91a73712c7f52fc0b1; no voters:
I20260811 17:14:36.482496 31444 leader_election.cc:288] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:14:36.482589 31449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:36.482821 31449 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [term 1 LEADER]: Becoming Leader. State: Replica: b8c1ceb0e5384e91a73712c7f52fc0b1, State: Running, Role: LEADER
I20260811 17:14:36.483191 31449 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35349 } }
I20260811 17:14:36.483887 31444 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:36.484395 31449 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8c1ceb0e5384e91a73712c7f52fc0b1. Latest consensus state: current_term: 1 leader_uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35349 } } }
I20260811 17:14:36.484625 31449 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:36.485423 31452 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35349 } } }
I20260811 17:14:36.485536 31452 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:36.485620 31455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:14:36.486110 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:36.469997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35516 (local address 127.21.54.190:35349)
0811 17:14:36.470439 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:36.470455 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:36.470492 (+ 37us) server_negotiation.cc:407] Connection header received
0811 17:14:36.471131 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:36.471149 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:36.471513 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:36.471821 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:36.472809 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:36.473930 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:36.475308 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:36.475814 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:36.475948 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:36.479054 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:36.479060 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:36.479075 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:36.479113 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:36.481263 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:36.481882 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:36.481888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:36.481895 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:36.482036 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:36.482388 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:36.482395 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:36.482399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:36.483060 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:36.483941 (+ 881us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:36.484128 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 17:14:36.484466 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":154,"threads_started":1}
I20260811 17:14:36.489113 31455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:14:36.495311 31455 catalog_manager.cc:1347] Generated new cluster ID: e9440f5783b34084a297e934c573f0b0
I20260811 17:14:36.495396 31455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:14:36.505079 31455 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:14:36.506145 31455 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:14:36.513332 31455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1: Generated new TSK 0
I20260811 17:14:36.514086 31455 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
I20260811 17:14:36.556195 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:35349
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:36.687703 31476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:36.688001 31476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:36.688048 31476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:36.692315 31476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:36.692396 31476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:36.692493 31476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:14:36.698429 31476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:35349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:36.699630 31476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:36.700795 31476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:36.711470 31482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:36.711695 31476 server_base.cc:1034] running on GCE node
W20260811 17:14:36.711472 31483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:36.711472 31486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:36.712273 31476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:36.713034 31476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:36.714252 31476 hybrid_clock.cc:648] HybridClock initialized: now 1786468476714231 us; error 32 us; skew 500 ppm
Aug 11 17:14:36 dist-test-slave-fd7l krb5kdc[31347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:36.717433 31476 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:14:36.718479 31476 webserver.cc:460] Webserver started at http://127.21.54.129:42481/ using document root <none> and password file <none>
I20260811 17:14:36.719066 31476 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:36.719130 31476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:36.719313 31476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:36.721616 31476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "d2bc2367de934c68927a8ff867ec417c"
format_stamp: "Formatted at 2026-08-11 17:14:36 on dist-test-slave-fd7l"
server_key: "99d1547df264fab1e0ec7e3a32a151d4"
server_key_iv: "5b8b8531a4895d6405597ce5743b54ae"
server_key_version: "encryptionkey@0"
I20260811 17:14:36.722213 31476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "d2bc2367de934c68927a8ff867ec417c"
format_stamp: "Formatted at 2026-08-11 17:14:36 on dist-test-slave-fd7l"
server_key: "99d1547df264fab1e0ec7e3a32a151d4"
server_key_iv: "5b8b8531a4895d6405597ce5743b54ae"
server_key_version: "encryptionkey@0"
I20260811 17:14:36.726399 31476 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:14:36.729269 31494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:36.730643 31476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 17:14:36.730762 31476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "d2bc2367de934c68927a8ff867ec417c"
format_stamp: "Formatted at 2026-08-11 17:14:36 on dist-test-slave-fd7l"
server_key: "99d1547df264fab1e0ec7e3a32a151d4"
server_key_iv: "5b8b8531a4895d6405597ce5743b54ae"
server_key_version: "encryptionkey@0"
I20260811 17:14:36.730860 31476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:36.754189 31476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:36.757967 31476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:36.758147 31476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:36.758725 31476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:36.759604 31476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:36.759657 31476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:36.759702 31476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:36.759718 31476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:36.769645 31476 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:42757
I20260811 17:14:36.769673 31634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:42757 every 8 connection(s)
I20260811 17:14:36.770648 31476 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:14:36.775745 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 31476
I20260811 17:14:36.775924 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:14:36.776278 21722 external_mini_cluster.cc:1428] Setting key b3fb7e57d84ed09bcac65410188b7bfe
Aug 11 17:14:36 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:36.786791 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:36.772736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:57381 (local address 127.21.54.190:35349)
0811 17:14:36.772882 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:36.772885 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:36.773858 (+ 973us) server_negotiation.cc:407] Connection header received
0811 17:14:36.775169 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:36.775173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:36.775228 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:36.775319 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:36.776802 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:36.777497 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:36.778830 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:36.779287 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:36.779429 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:36.782300 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:36.782304 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:36.782306 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:36.782333 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:36.784141 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:36.784905 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:36.784915 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:36.784916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:36.785006 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:36.785428 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:36.785431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:36.785432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:36.785637 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:36.785782 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:36.786453 (+ 671us) server_negotiation.cc:299] Negotiation successful
0811 17:14:36.786602 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 17:14:36.787746 31637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:36.773167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.129:57381)
0811 17:14:36.773675 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:36.773721 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:36.774933 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:36.775505 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:36.775512 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:36.775943 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:36.776611 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:36.776629 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:36.777654 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:36.777658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:36.778680 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:36.778692 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:36.779452 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:36.779455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:36.779603 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:36.780294 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:36.780312 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:36.782080 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:36.784362 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:36.784368 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:36.784383 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:36.784776 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:36.785108 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:36.785111 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:36.785113 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:36.785274 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:36.785791 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:36.785801 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:36.786157 (+ 356us) client_negotiation.cc:769] Sending connection context
0811 17:14:36.786369 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 17:14:36.786755 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":139,"threads_started":1}
WARNING: no policy specified for kudu/127.21.54.130@KRBTEST.COM; defaulting to no policy
I20260811 17:14:36.789286 31635 heartbeater.cc:347] Connected to a master server at 127.21.54.190:35349
I20260811 17:14:36.789595 31635 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:36.790205 31635 heartbeater.cc:511] Master 127.21.54.190:35349 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.130@KRBTEST.COM" created.
I20260811 17:14:36.792093 31400 ts_manager.cc:194] Registered new tserver with Master: d2bc2367de934c68927a8ff867ec417c (127.21.54.129:42757)
I20260811 17:14:36.793373 31400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:57381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
I20260811 17:14:36.812959 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:35349
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:36.935994 31641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:36.936328 31641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:36.936407 31641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:36.940734 31641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:36.940831 31641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:36.940922 31641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:14:36.946532 31641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:35349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:36.947752 31641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:36.948874 31641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:36.959195 31650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:36.959211 31652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:36.959211 31649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:36.959316 31641 server_base.cc:1034] running on GCE node
I20260811 17:14:36.959964 31641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:36.960855 31641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:36.962070 31641 hybrid_clock.cc:648] HybridClock initialized: now 1786468476962048 us; error 34 us; skew 500 ppm
Aug 11 17:14:36 dist-test-slave-fd7l krb5kdc[31347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:36.965234 31641 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:14:36.966138 31641 webserver.cc:460] Webserver started at http://127.21.54.130:37093/ using document root <none> and password file <none>
I20260811 17:14:36.966696 31641 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:36.966756 31641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:36.966934 31641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:36.969131 31641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "ac09c315eb83447aa348570fb86d1324"
format_stamp: "Formatted at 2026-08-11 17:14:36 on dist-test-slave-fd7l"
server_key: "3bd29c8060fdbf32549e7c5567eb2a24"
server_key_iv: "c7d1ffa5555241f4dfb34fed9f3f6b00"
server_key_version: "encryptionkey@0"
I20260811 17:14:36.969691 31641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "ac09c315eb83447aa348570fb86d1324"
format_stamp: "Formatted at 2026-08-11 17:14:36 on dist-test-slave-fd7l"
server_key: "3bd29c8060fdbf32549e7c5567eb2a24"
server_key_iv: "c7d1ffa5555241f4dfb34fed9f3f6b00"
server_key_version: "encryptionkey@0"
I20260811 17:14:36.973901 31641 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:14:36.976660 31661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:36.978063 31641 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:36.978223 31641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "ac09c315eb83447aa348570fb86d1324"
format_stamp: "Formatted at 2026-08-11 17:14:36 on dist-test-slave-fd7l"
server_key: "3bd29c8060fdbf32549e7c5567eb2a24"
server_key_iv: "c7d1ffa5555241f4dfb34fed9f3f6b00"
server_key_version: "encryptionkey@0"
I20260811 17:14:36.978348 31641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:36.998960 31641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:37.002691 31641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:37.002933 31641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:37.003680 31641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:37.004670 31641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:37.004722 31641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:37.004765 31641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:37.004781 31641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:37.014676 31641 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:39989
I20260811 17:14:37.014695 31792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:39989 every 8 connection(s)
I20260811 17:14:37.015654 31641 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:14:37.021865 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 31641
I20260811 17:14:37.022035 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:14:37.022379 21722 external_mini_cluster.cc:1428] Setting key 11f8b6aa4ad795187eb4567f4dc1000e
Aug 11 17:14:37 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468476, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:37.032229 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:37.017635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:47369 (local address 127.21.54.190:35349)
0811 17:14:37.017809 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:37.017813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:37.018639 (+ 826us) server_negotiation.cc:407] Connection header received
0811 17:14:37.020016 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:37.020019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:37.020073 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:37.020164 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:37.021665 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.022548 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.024360 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.024735 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.024879 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:37.028198 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:37.028202 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:37.028204 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:37.028231 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:37.029900 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:37.030525 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:37.030529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:37.030530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:37.030615 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:37.031003 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:37.031007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:37.031009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:37.031217 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:37.031332 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:37.031893 (+ 561us) server_negotiation.cc:299] Negotiation successful
0811 17:14:37.032046 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 17:14:37.033187 31796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:37.018036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.130:47369)
0811 17:14:37.018469 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:37.018513 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:37.019741 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:37.020381 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:37.020388 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:37.020933 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:37.021461 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:37.021475 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.022728 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.022732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:37.024199 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:37.024212 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.024917 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.024924 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:37.025198 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:37.025961 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:37.025989 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:37.027968 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:37.030066 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:37.030073 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:37.030092 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:37.030386 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:37.030734 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:37.030738 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:37.030740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:37.030897 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:37.031331 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:37.031340 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:37.031639 (+ 299us) client_negotiation.cc:769] Sending connection context
0811 17:14:37.031849 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 17:14:37.032205 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for kudu/127.21.54.131@KRBTEST.COM; defaulting to no policy
I20260811 17:14:37.034615 31793 heartbeater.cc:347] Connected to a master server at 127.21.54.190:35349
I20260811 17:14:37.034895 31793 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:37.035507 31793 heartbeater.cc:511] Master 127.21.54.190:35349 requested a full tablet report, sending...
I20260811 17:14:37.036852 31400 ts_manager.cc:194] Registered new tserver with Master: ac09c315eb83447aa348570fb86d1324 (127.21.54.130:39989)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.131@KRBTEST.COM" created.
I20260811 17:14:37.037652 31400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:47369
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
I20260811 17:14:37.057055 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:35349
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:37.199573 31800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:37.200054 31800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:37.200155 31800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:37.205156 31800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:37.205227 31800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:37.205313 31800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:14:37.211108 31800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:35349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:37.212283 31800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:37.213440 31800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:14:37.223990 31800 server_base.cc:1034] running on GCE node
W20260811 17:14:37.223942 31806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:37.223942 31809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:37.223942 31807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:37.224762 31800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:37.225576 31800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:37.226806 31800 hybrid_clock.cc:648] HybridClock initialized: now 1786468477226790 us; error 29 us; skew 500 ppm
Aug 11 17:14:37 dist-test-slave-fd7l krb5kdc[31347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468477, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:37.230306 31800 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:14:37.231360 31800 webserver.cc:460] Webserver started at http://127.21.54.131:35837/ using document root <none> and password file <none>
I20260811 17:14:37.232067 31800 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:37.232172 31800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:37.232453 31800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:37.234967 31800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "66a9506367f1435ba9b0a423c774f503"
format_stamp: "Formatted at 2026-08-11 17:14:37 on dist-test-slave-fd7l"
server_key: "29bfa4b9b8837ae17f050ffda3e2d5e3"
server_key_iv: "e823224523cab68cf899b45d0451241f"
server_key_version: "encryptionkey@0"
I20260811 17:14:37.235720 31800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "66a9506367f1435ba9b0a423c774f503"
format_stamp: "Formatted at 2026-08-11 17:14:37 on dist-test-slave-fd7l"
server_key: "29bfa4b9b8837ae17f050ffda3e2d5e3"
server_key_iv: "e823224523cab68cf899b45d0451241f"
server_key_version: "encryptionkey@0"
I20260811 17:14:37.240267 31800 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 17:14:37.243119 31817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:37.244709 31800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 17:14:37.244848 31800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "66a9506367f1435ba9b0a423c774f503"
format_stamp: "Formatted at 2026-08-11 17:14:37 on dist-test-slave-fd7l"
server_key: "29bfa4b9b8837ae17f050ffda3e2d5e3"
server_key_iv: "e823224523cab68cf899b45d0451241f"
server_key_version: "encryptionkey@0"
I20260811 17:14:37.244969 31800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:37.273419 31800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:37.277581 31800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:37.277820 31800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:37.278520 31800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:37.279502 31800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:37.279592 31800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:37.279698 31800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:37.279754 31800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:37.290941 31800 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:35199
I20260811 17:14:37.290963 31946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:35199 every 8 connection(s)
I20260811 17:14:37.292018 31800 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:14:37.296561 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 31800
I20260811 17:14:37.296715 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:14:37.297060 21722 external_mini_cluster.cc:1428] Setting key 03958e9392a950cb552f25d789c8ffc9
Aug 11 17:14:37 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468477, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:37.308931 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:37.294168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:39131 (local address 127.21.54.190:35349)
0811 17:14:37.294345 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:37.294349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:37.295487 (+ 1138us) server_negotiation.cc:407] Connection header received
0811 17:14:37.297500 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:37.297504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:37.297568 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:37.297707 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:37.299331 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.299940 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.301271 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.301512 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.301633 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:37.304596 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:37.304600 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:37.304602 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:37.304631 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:37.306503 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:37.307139 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:37.307142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:37.307143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:37.307234 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:37.307610 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:37.307613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:37.307614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:37.307837 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:37.307937 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:37.308660 (+ 723us) server_negotiation.cc:299] Negotiation successful
0811 17:14:37.308796 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 17:14:37.309798 31951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:37.294659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.131:39131)
0811 17:14:37.295260 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:37.295309 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:37.297238 (+ 1929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:37.297902 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:37.297912 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:37.298506 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:37.299174 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:37.299190 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.300080 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.300084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:37.301133 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:37.301143 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.301679 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.301684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:37.301878 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:37.302557 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:37.302575 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:37.304325 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:37.306674 (+ 2349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:37.306680 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:37.306697 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:37.307007 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:37.307348 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:37.307350 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:37.307352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:37.307512 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:37.307954 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:37.307960 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:37.308267 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 17:14:37.308447 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 17:14:37.308835 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":198,"threads_started":1}
I20260811 17:14:37.311002 31949 heartbeater.cc:347] Connected to a master server at 127.21.54.190:35349
I20260811 17:14:37.311314 31949 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:37.311960 31949 heartbeater.cc:511] Master 127.21.54.190:35349 requested a full tablet report, sending...
I20260811 17:14:37.313220 31399 ts_manager.cc:194] Registered new tserver with Master: 66a9506367f1435ba9b0a423c774f503 (127.21.54.131:35199)
I20260811 17:14:37.313877 31399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:39131
I20260811 17:14:37.324340 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.129.keytab.
Entry for principal oryx/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.129.keytab.
WARNING: no policy specified for oryx/127.21.54.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.54.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.130.keytab.
Entry for principal oryx/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.130.keytab.
WARNING: no policy specified for oryx/127.21.54.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.54.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.131.keytab.
Entry for principal oryx/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.131.keytab.
I20260811 17:14:37.419963 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 31476
I20260811 17:14:37.430122 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 31641
I20260811 17:14:37.439390 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 31800
I20260811 17:14:37.452657 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 31362
I20260811 17:14:37.461910 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:35349
--webserver_interface=127.21.54.190
--webserver_port=33917
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:35349
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:37.582859 31960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:37.583181 31960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:37.583230 31960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:37.587370 31960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:37.587445 31960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:37.587466 31960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:37.587483 31960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:37.587498 31960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:14:37.593132 31960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:35349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:35349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=33917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:37.594281 31960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:37.595326 31960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:37.604558 31971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:37.604635 31969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:37.604635 31967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:37.604872 31960 server_base.cc:1034] running on GCE node
I20260811 17:14:37.605506 31960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:37.606709 31960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:37.607933 31960 hybrid_clock.cc:648] HybridClock initialized: now 1786468477607921 us; error 35 us; skew 500 ppm
Aug 11 17:14:37 dist-test-slave-fd7l krb5kdc[31347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468477, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:37.610934 31960 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:14:37.611938 31960 webserver.cc:460] Webserver started at http://127.21.54.190:33917/ using document root <none> and password file <none>
I20260811 17:14:37.612609 31960 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:37.612731 31960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:37.616897 31960 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 17:14:37.619241 31979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:37.620731 31960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:37.620899 31960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1"
format_stamp: "Formatted at 2026-08-11 17:14:36 on dist-test-slave-fd7l"
server_key: "624957181d3b0f002ead9d14349b0c25"
server_key_iv: "a1ec28b8f213d7509cda7da7ebefaaba"
server_key_version: "encryptionkey@0"
I20260811 17:14:37.621572 31960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:37.634354 31960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:37.638140 31960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:37.638392 31960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:37.646798 31960 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:35349
I20260811 17:14:37.646809 32038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:35349 every 8 connection(s)
I20260811 17:14:37.648007 31960 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:14:37.652065 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 31960
I20260811 17:14:37.652598 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:42757
--local_ip_for_outbound_sockets=127.21.54.129
--tserver_master_addrs=127.21.54.190:35349
--webserver_port=42481
--webserver_interface=127.21.54.129
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.54.129
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.129.keytab with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:14:37.652786 32039 sys_catalog.cc:263] Verifying existing consensus state
I20260811 17:14:37.654745 32039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1: Bootstrap starting.
I20260811 17:14:37.672667 32039 log.cc:824] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:37.680295 32039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:14:37.680838 32039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1: Bootstrap complete.
I20260811 17:14:37.685511 32039 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35349 } }
I20260811 17:14:37.685973 32039 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c1ceb0e5384e91a73712c7f52fc0b1, State: Initialized, Role: FOLLOWER
I20260811 17:14:37.686582 32039 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35349 } }
I20260811 17:14:37.686736 32039 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:14:37.686836 32039 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:14:37.686964 32039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [term 1 FOLLOWER]: Advancing to term 2
I20260811 17:14:37.689291 32039 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35349 } }
I20260811 17:14:37.689677 32039 leader_election.cc:302] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8c1ceb0e5384e91a73712c7f52fc0b1; no voters:
I20260811 17:14:37.690024 32039 leader_election.cc:288] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 17:14:37.690181 32044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 17:14:37.690450 32044 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [term 2 LEADER]: Becoming Leader. State: Replica: b8c1ceb0e5384e91a73712c7f52fc0b1, State: Running, Role: LEADER
I20260811 17:14:37.690804 32044 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35349 } }
I20260811 17:14:37.691432 32039 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:37.692483 32046 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8c1ceb0e5384e91a73712c7f52fc0b1. Latest consensus state: current_term: 2 leader_uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35349 } } }
I20260811 17:14:37.692545 32045 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c1ceb0e5384e91a73712c7f52fc0b1" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35349 } } }
I20260811 17:14:37.692675 32045 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:37.692677 32046 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:37.696780 32060 catalog_manager.cc:1259] Loaded cluster ID: e9440f5783b34084a297e934c573f0b0
I20260811 17:14:37.696871 32060 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1: loading cluster ID for follower catalog manager: success
I20260811 17:14:37.698856 32060 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1: acquiring CA information for follower catalog manager: success
I20260811 17:14:37.700165 32060 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 17:14:37.700304 32054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:14:37.701085 32054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:14:37.701521 32054 catalog_manager.cc:1259] Loaded cluster ID: e9440f5783b34084a297e934c573f0b0
I20260811 17:14:37.701596 32054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:14:37.702162 32054 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:14:37.702574 32054 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b8c1ceb0e5384e91a73712c7f52fc0b1: Loaded TSK: 0
I20260811 17:14:37.703274 32054 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 17:14:37.783075 32042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:37.783352 32042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:37.783401 32042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:37.787675 32042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:37.787748 32042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:37.787844 32042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:14:37.793653 32042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.129.keytab
--principal=oryx/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:42757
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=42481
--tserver_master_addrs=127.21.54.190:35349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:37.794852 32042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:37.796012 32042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:37.807458 32068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:37.807447 32070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:37.807451 32067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:37.808018 32042 server_base.cc:1034] running on GCE node
I20260811 17:14:37.808477 32042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:37.809221 32042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:37.810421 32042 hybrid_clock.cc:648] HybridClock initialized: now 1786468477810391 us; error 48 us; skew 500 ppm
Aug 11 17:14:37 dist-test-slave-fd7l krb5kdc[31347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468477, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:37.813594 32042 init.cc:377] Logged in from keytab as oryx/127.21.54.129@KRBTEST.COM (short username oryx)
I20260811 17:14:37.814689 32042 webserver.cc:460] Webserver started at http://127.21.54.129:42481/ using document root <none> and password file <none>
I20260811 17:14:37.815321 32042 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:37.815402 32042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:37.819999 32042 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260811 17:14:37.822455 32077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:37.823916 32042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:14:37.824079 32042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "d2bc2367de934c68927a8ff867ec417c"
format_stamp: "Formatted at 2026-08-11 17:14:36 on dist-test-slave-fd7l"
server_key: "99d1547df264fab1e0ec7e3a32a151d4"
server_key_iv: "5b8b8531a4895d6405597ce5743b54ae"
server_key_version: "encryptionkey@0"
I20260811 17:14:37.824728 32042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:37.837024 32042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:37.840452 32042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:37.840695 32042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:37.841419 32042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:37.842301 32042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:37.842383 32042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:37.842455 32042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:37.842501 32042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:37.852087 32042 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:42757
I20260811 17:14:37.852152 32214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:42757 every 8 connection(s)
I20260811 17:14:37.853261 32042 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:14:37.861637 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 32042
I20260811 17:14:37.862237 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:39989
--local_ip_for_outbound_sockets=127.21.54.130
--tserver_master_addrs=127.21.54.190:35349
--webserver_port=37093
--webserver_interface=127.21.54.130
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.54.130
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.130.keytab with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 17:14:37 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:37 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:37.868253 32220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:37.855461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.129:51947)
0811 17:14:37.855922 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:37.855977 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:37.857345 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:37.858571 (+ 1226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:37.858578 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:37.858950 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:37.859438 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:37.859452 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.860845 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.860848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:37.862074 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:37.862089 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.863091 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.863098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:37.863329 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:37.864057 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:37.864085 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:37.866981 (+ 2896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":140,"threads_started":1}
W20260811 17:14:37.869573 32219 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:37.855414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:51947 (local address 127.21.54.190:35349)
0811 17:14:37.855880 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:37.855896 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:37.856105 (+ 209us) server_negotiation.cc:407] Connection header received
0811 17:14:37.857787 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:37.857798 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:37.858137 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:37.858489 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:37.859623 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.860706 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.862233 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.862898 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.863044 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:37.867976 (+ 4932us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.129:51947: BlockingRecv error: recv got EOF from 127.21.54.129:51947 (error 108)
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":142,"threads_started":1}
W20260811 17:14:37.872892 32215 heartbeater.cc:644] Failed to heartbeat to 127.21.54.190:35349 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.54.190:35349: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Aug 11 17:14:37 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:37 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:37.883960 32220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:37.873562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.129:52523)
0811 17:14:37.873743 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:37.873761 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:37.873886 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:37.877531 (+ 3645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:37.877535 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:37.877802 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:37.877989 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:37.878002 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.879116 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.879121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:37.880072 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:37.880087 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.880988 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.880994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:37.881106 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:37.881663 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:37.881678 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:37.883836 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 17:14:37.884235 32219 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:37.876798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:52523 (local address 127.21.54.190:35349)
0811 17:14:37.877206 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:37.877211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:37.877232 (+ 21us) server_negotiation.cc:407] Connection header received
0811 17:14:37.877292 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:37.877296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:37.877358 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:37.877464 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:37.878157 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.878963 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.880346 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.880758 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.880866 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:37.884060 (+ 3194us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.129:52523: BlockingRecv error: recv got EOF from 127.21.54.129:52523 (error 108)
Metrics: {"server-negotiator.queue_time_us":300}
Aug 11 17:14:37 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:37 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:37.890074 32220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:37.884747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.129:40039)
0811 17:14:37.884870 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:37.884886 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:37.884974 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:37.885263 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:37.885267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:37.885440 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:37.885520 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:37.885525 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.886301 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.886306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:37.886852 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:37.886861 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.887350 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.887353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:37.887439 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:37.887981 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:37.887994 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:37.889910 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 17:14:37.890197 32219 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:37.884882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:40039 (local address 127.21.54.190:35349)
0811 17:14:37.885035 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:37.885041 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:37.885056 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:14:37.885103 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:37.885105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:37.885154 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:37.885244 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:37.885648 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.886182 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.886982 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:37.887228 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:37.887343 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:37.890048 (+ 2705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.129:40039: BlockingRecv error: recv got EOF from 127.21.54.129:40039 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 17:14:37.890443 32215 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 17:14:37.991750 32221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:37.992100 32221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:37.992203 32221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:37.996556 32221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:37.996630 32221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:37.996693 32221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:14:38.002444 32221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.130.keytab
--principal=oryx/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:39989
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=37093
--tserver_master_addrs=127.21.54.190:35349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:38.003577 32221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:38.004629 32221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:38.015066 32229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:38.015144 32221 server_base.cc:1034] running on GCE node
W20260811 17:14:38.015053 32226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:38.015058 32227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:38.015899 32221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:38.016697 32221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:38.017907 32221 hybrid_clock.cc:648] HybridClock initialized: now 1786468478017866 us; error 54 us; skew 500 ppm
Aug 11 17:14:38 dist-test-slave-fd7l krb5kdc[31347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468478, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:38.021530 32221 init.cc:377] Logged in from keytab as oryx/127.21.54.130@KRBTEST.COM (short username oryx)
I20260811 17:14:38.022682 32221 webserver.cc:460] Webserver started at http://127.21.54.130:37093/ using document root <none> and password file <none>
I20260811 17:14:38.023310 32221 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:38.023514 32221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:38.028239 32221 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:14:38.030827 32238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:38.032292 32221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 17:14:38.032450 32221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "ac09c315eb83447aa348570fb86d1324"
format_stamp: "Formatted at 2026-08-11 17:14:36 on dist-test-slave-fd7l"
server_key: "3bd29c8060fdbf32549e7c5567eb2a24"
server_key_iv: "c7d1ffa5555241f4dfb34fed9f3f6b00"
server_key_version: "encryptionkey@0"
I20260811 17:14:38.033159 32221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:38.049691 32221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:38.053416 32221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:38.053644 32221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:38.054320 32221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:38.055248 32221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:38.055323 32221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:38.055397 32221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:38.055439 32221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:38.065694 32221 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:39989
I20260811 17:14:38.065722 32373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:39989 every 8 connection(s)
I20260811 17:14:38.066866 32221 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:14:38.071813 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 32221
I20260811 17:14:38.072302 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:35199
--local_ip_for_outbound_sockets=127.21.54.131
--tserver_master_addrs=127.21.54.190:35349
--webserver_port=35837
--webserver_interface=127.21.54.131
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.54.131
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.131.keytab with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 17:14:38 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:38 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 17:14:38.079584 32219 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:38.069073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:46597 (local address 127.21.54.190:35349)
0811 17:14:38.069216 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:38.069220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:38.070080 (+ 860us) server_negotiation.cc:407] Connection header received
0811 17:14:38.071445 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:38.071449 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:38.071515 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:38.071643 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:38.073099 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.073639 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.074829 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.075121 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.075246 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:38.079392 (+ 4146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.130:46597: BlockingRecv error: recv got EOF from 127.21.54.130:46597 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 17:14:38.080111 32376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:38.069436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.130:46597)
0811 17:14:38.069901 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:38.069948 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:38.071216 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:38.071843 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:38.071850 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:38.072279 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:38.072938 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:38.072953 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.073776 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.073784 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:38.074692 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:38.074701 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.075316 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.075320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:38.075498 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:38.076220 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:38.076239 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:38.079133 (+ 2894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":122,"threads_started":1}
W20260811 17:14:38.080816 32374 heartbeater.cc:644] Failed to heartbeat to 127.21.54.190:35349 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.54.190:35349: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Aug 11 17:14:38 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:38 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:38.088284 32376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:38.081397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.130:35073)
0811 17:14:38.081651 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:38.081676 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:38.081783 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:38.082162 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:38.082165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:38.082324 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:38.082431 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:38.082437 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.083606 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.083609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:38.084198 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:38.084208 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.084945 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.084948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:38.085034 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:38.085705 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:38.085718 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:38.088062 (+ 2344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177}
W20260811 17:14:38.088398 32219 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:38.081637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:35073 (local address 127.21.54.190:35349)
0811 17:14:38.081810 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:38.081818 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:38.081859 (+ 41us) server_negotiation.cc:407] Connection header received
0811 17:14:38.081917 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:38.081923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:38.081988 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:38.082104 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:38.082567 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.083457 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.084340 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.084796 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.084936 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:38.088264 (+ 3328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.130:35073: BlockingRecv error: recv got EOF from 127.21.54.130:35073 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 17:14:38 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:38 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:38.095415 32376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:38.088921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.130:50535)
0811 17:14:38.089215 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:38.089236 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:38.089356 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:38.089672 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:38.089675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:38.089825 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:38.089912 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:38.089917 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.090988 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.090991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:38.091713 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:38.091724 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.092340 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.092343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:38.092575 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:38.093194 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:38.093208 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:38.095296 (+ 2088us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200}
W20260811 17:14:38.095721 32374 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 17:14:38.095935 32219 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:38.089059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:50535 (local address 127.21.54.190:35349)
0811 17:14:38.089280 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:38.089284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:38.089302 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:14:38.089467 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:38.089470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:38.089519 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:38.089606 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:38.090021 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.090841 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.091835 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.092198 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.092301 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:38.095468 (+ 3167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.130:50535: BlockingRecv error: recv got EOF from 127.21.54.130:50535 (error 108)
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":127,"threads_started":1}
W20260811 17:14:38.201279 32377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:38.201720 32377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:38.201808 32377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:38.206868 32377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:38.206955 32377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:38.207027 32377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:14:38.213757 32377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/krb5kdc/oryx_127.21.54.131.keytab
--principal=oryx/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:35199
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=35837
--tserver_master_addrs=127.21.54.190:35349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:38.214943 32377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:38.216066 32377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:38.227119 32390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:38.227065 32384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:38.227059 32386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:38.227675 32377 server_base.cc:1034] running on GCE node
I20260811 17:14:38.228099 32377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:38.228909 32377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:38.230120 32377 hybrid_clock.cc:648] HybridClock initialized: now 1786468478230108 us; error 34 us; skew 500 ppm
Aug 11 17:14:38 dist-test-slave-fd7l krb5kdc[31347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468478, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:38.233373 32377 init.cc:377] Logged in from keytab as oryx/127.21.54.131@KRBTEST.COM (short username oryx)
I20260811 17:14:38.234396 32377 webserver.cc:460] Webserver started at http://127.21.54.131:35837/ using document root <none> and password file <none>
I20260811 17:14:38.235059 32377 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:38.235179 32377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:38.239693 32377 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:14:38.242300 32397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:38.243841 32377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:14:38.244000 32377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "66a9506367f1435ba9b0a423c774f503"
format_stamp: "Formatted at 2026-08-11 17:14:37 on dist-test-slave-fd7l"
server_key: "29bfa4b9b8837ae17f050ffda3e2d5e3"
server_key_iv: "e823224523cab68cf899b45d0451241f"
server_key_version: "encryptionkey@0"
I20260811 17:14:38.244740 32377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:38.268270 32377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:38.272184 32377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:38.272433 32377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:38.273180 32377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:38.274163 32377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:38.274243 32377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:38.274325 32377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:38.274372 32377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:38.284869 32377 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:35199
I20260811 17:14:38.284905 32537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:35199 every 8 connection(s)
I20260811 17:14:38.285919 32377 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:14:38.292222 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 32377
Aug 11 17:14:38 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:38 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 17:14:38.300702 32219 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:38.287837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:55931 (local address 127.21.54.190:35349)
0811 17:14:38.287992 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:38.287996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:38.289018 (+ 1022us) server_negotiation.cc:407] Connection header received
0811 17:14:38.290593 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:38.290597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:38.290667 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:38.290807 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:38.292287 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.293229 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.294986 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.295208 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.295293 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:38.300427 (+ 5134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.131:55931: BlockingRecv error: recv got EOF from 127.21.54.131:55931 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:14:38.301440 32540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:38.288224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.131:55931)
0811 17:14:38.288818 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:38.288874 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:38.290283 (+ 1409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:38.291020 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:38.291028 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:38.291450 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:38.292098 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:38.292127 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.293469 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.293475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:38.294835 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:38.294849 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.295384 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.295391 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:38.295619 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:38.296335 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:38.296364 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:38.300208 (+ 3844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":112,"threads_started":1}
W20260811 17:14:38.302116 32538 heartbeater.cc:644] Failed to heartbeat to 127.21.54.190:35349 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.54.190:35349: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
I20260811 17:14:38.304335 32378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:38.293407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35530 (local address 127.21.54.190:35349)
0811 17:14:38.293585 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:38.293591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:38.293608 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:14:38.293653 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:38.293656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:38.293710 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:38.293820 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:38.294560 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.295341 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.296376 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.296907 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.297053 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:38.298553 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:38.298566 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:38.298591 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:38.298637 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:38.301080 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:38.301627 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:38.301637 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:38.301647 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:38.301838 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:38.302361 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:38.302366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:38.302368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:38.303002 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:38.303228 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:38.303856 (+ 628us) server_negotiation.cc:299] Negotiation successful
0811 17:14:38.304164 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 17:14:38 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:38 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:38.310596 32540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:38.302699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.131:35807)
0811 17:14:38.302830 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:38.302848 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:38.302939 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:38.304864 (+ 1925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:38.304871 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:38.305093 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:38.305257 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:38.305269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.306417 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.306420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:38.307095 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:38.307105 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.307704 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.307708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:38.307805 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:38.308401 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:38.308415 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:38.310480 (+ 2065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 17:14:38.311141 32219 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:38.304253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:35807 (local address 127.21.54.190:35349)
0811 17:14:38.304430 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:38.304435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:38.304453 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:14:38.304498 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:38.304501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:38.304588 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:38.304698 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:38.305526 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.306271 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.307209 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.307546 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.307636 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:38.311009 (+ 3373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.131:35807: BlockingRecv error: recv got EOF from 127.21.54.131:35807 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 17:14:38 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:38 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:38.317425 32540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:38.311363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.131:33019)
0811 17:14:38.311599 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:38.311615 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:38.311713 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:38.312080 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:38.312083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:38.312240 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:38.312331 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:38.312336 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.313299 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.313301 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:38.313888 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:38.313897 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.314376 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.314378 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:38.314440 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:38.315013 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:38.315026 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:38.317256 (+ 2230us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119}
W20260811 17:14:38.317510 32219 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:38.311620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:33019 (local address 127.21.54.190:35349)
0811 17:14:38.311803 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:38.311807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:38.311822 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:14:38.311859 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:38.311861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:38.311903 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:38.311977 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:38.312473 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.313171 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.314023 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.314260 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.314352 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:38.317389 (+ 3037us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.131:33019: BlockingRecv error: recv got EOF from 127.21.54.131:33019 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
W20260811 17:14:38.317796 32538 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 17:14:38 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:38 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:38.897815 32542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:38.891260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.129:36557)
0811 17:14:38.891559 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:38.891576 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:38.891748 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:38.892079 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:38.892081 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:38.892308 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:38.892411 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:38.892417 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.893444 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.893450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:38.894340 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:38.894353 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.894846 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.894848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:38.894912 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:38.895484 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:38.895496 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:38.897566 (+ 2070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":84,"threads_started":1}
W20260811 17:14:38.897838 32543 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:38.891393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:36557 (local address 127.21.54.190:35349)
0811 17:14:38.891797 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:38.891801 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:38.891817 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:14:38.891865 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:38.891867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:38.891913 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:38.892002 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:38.892596 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.893329 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.894469 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:38.894733 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:38.894822 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:38.897714 (+ 2892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.129:36557: BlockingRecv error: recv got EOF from 127.21.54.129:36557 (error 108)
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":86,"threads_started":1}
Aug 11 17:14:39 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:39 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:39.102895 32544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:39.096474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.130:45789)
0811 17:14:39.096832 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:39.096847 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:39.096955 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:39.097234 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:39.097236 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:39.097429 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:39.097532 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:39.097538 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:39.098387 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:39.098389 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:39.098980 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:39.098989 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:39.099595 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:39.099597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:39.099692 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:39.100297 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:39.100311 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:39.102692 (+ 2381us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":117,"threads_started":1}
W20260811 17:14:39.103006 32543 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:39.096657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:45789 (local address 127.21.54.190:35349)
0811 17:14:39.096817 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:39.096821 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:39.096928 (+ 107us) server_negotiation.cc:407] Connection header received
0811 17:14:39.097047 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:39.097050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:39.097104 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:39.097194 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:39.097672 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:39.098278 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:39.099121 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:39.099412 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:39.099567 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:39.102879 (+ 3312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.130:45789: BlockingRecv error: recv got EOF from 127.21.54.130:45789 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 17:14:39 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:39 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:39.325124 32546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:39.318622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.131:34563)
0811 17:14:39.318983 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:39.319000 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:39.319163 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:39.319460 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:39.319463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:39.319693 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:39.319788 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:39.319794 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:39.320656 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:39.320659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:39.321559 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:39.321573 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:39.322108 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:39.322111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:39.322190 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:39.322830 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:39.322845 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:39.324921 (+ 2076us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":131,"threads_started":1}
W20260811 17:14:39.325217 32543 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:39.318835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:34563 (local address 127.21.54.190:35349)
0811 17:14:39.318985 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:39.318991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:39.319096 (+ 105us) server_negotiation.cc:407] Connection header received
0811 17:14:39.319275 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:39.319278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:39.319336 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:39.319436 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:39.319917 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:39.320494 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:39.321692 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:39.321974 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:39.322065 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:39.325089 (+ 3024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.131:34563: BlockingRecv error: recv got EOF from 127.21.54.131:34563 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 17:14:39 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:39 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:39.905471 32547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:39.898893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.129:59453)
0811 17:14:39.899266 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:39.899282 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:39.899395 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:39.899751 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:39.899754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:39.899967 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:39.900077 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:39.900083 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:39.900968 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:39.900971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:39.901645 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:39.901655 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:39.902119 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:39.902122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:39.902204 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:39.902739 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:39.902751 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:39.905265 (+ 2514us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":98,"threads_started":1}
W20260811 17:14:39.905539 32548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:39.899034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:59453 (local address 127.21.54.190:35349)
0811 17:14:39.899330 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:39.899334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:39.899351 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:14:39.899509 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:39.899512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:39.899562 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:39.899681 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:39.900190 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:39.900848 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:39.901772 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:39.902007 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:39.902097 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:39.905406 (+ 3309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.129:59453: BlockingRecv error: recv got EOF from 127.21.54.129:59453 (error 108)
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":113,"threads_started":1}
Aug 11 17:14:40 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:40 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:40.111161 32549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:40.104049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.130:39347)
0811 17:14:40.105093 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:40.105112 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:40.105276 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:40.105762 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:40.105765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:40.106032 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:40.106159 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:40.106166 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:40.107020 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:40.107022 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:40.107597 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:40.107606 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:40.108149 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:40.108152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:40.108225 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:40.108838 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:40.108850 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:40.111006 (+ 2156us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":909,"thread_start_us":147,"threads_started":1}
W20260811 17:14:40.111518 32548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:40.104127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:39347 (local address 127.21.54.190:35349)
0811 17:14:40.104301 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:40.104308 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:40.105216 (+ 908us) server_negotiation.cc:407] Connection header received
0811 17:14:40.105415 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:40.105420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:40.105482 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:40.105563 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:40.106318 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:40.106878 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:40.107764 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:40.107999 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:40.108131 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:40.111208 (+ 3077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.130:39347: BlockingRecv error: recv got EOF from 127.21.54.130:39347 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 17:14:40 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:40 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:40.333227 32550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:40.326212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.131:50325)
0811 17:14:40.326663 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:40.326680 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:40.326833 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:40.327184 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:40.327188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:40.327396 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:40.327497 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:40.327503 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:40.328590 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:40.328593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:40.329379 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:40.329389 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:40.330010 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:40.330013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:40.330088 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:40.330697 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:40.330710 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:40.333064 (+ 2354us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":84,"threads_started":1}
W20260811 17:14:40.333436 32548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:40.326422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:50325 (local address 127.21.54.190:35349)
0811 17:14:40.326558 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:40.326563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:40.326779 (+ 216us) server_negotiation.cc:407] Connection header received
0811 17:14:40.326961 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:40.326964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:40.327029 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:40.327141 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:40.327640 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:40.328412 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:40.329507 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:40.329857 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:40.329956 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:40.333254 (+ 3298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.131:50325: BlockingRecv error: recv got EOF from 127.21.54.131:50325 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 11 17:14:40 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:40 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:40.912340 32552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:40.906320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.129:36001)
0811 17:14:40.906682 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:40.906698 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:40.906792 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:40.907082 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:40.907084 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:40.907267 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:40.907366 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:40.907372 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:40.908171 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:40.908174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:40.908808 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:40.908819 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:40.909357 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:40.909360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:40.909432 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:40.910000 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:40.910012 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:40.912170 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":93,"threads_started":1}
W20260811 17:14:40.912513 32553 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:40.906440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:36001 (local address 127.21.54.190:35349)
0811 17:14:40.906747 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:40.906751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:40.906766 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:14:40.906885 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:40.906887 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:40.906933 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:40.907012 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:40.907473 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:40.908039 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:40.908953 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:40.909227 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:40.909325 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:40.912359 (+ 3034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.129:36001: BlockingRecv error: recv got EOF from 127.21.54.129:36001 (error 108)
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":151,"threads_started":1}
Aug 11 17:14:41 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:41 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:41.118561 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:41.112125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.130:33213)
0811 17:14:41.112460 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:41.112477 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:41.112671 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:41.112984 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:41.112988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:41.113233 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:41.113336 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:41.113343 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:41.114215 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:41.114220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:41.114842 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:41.114851 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:41.115373 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:41.115377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:41.115473 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:41.116068 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:41.116082 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:41.118295 (+ 2213us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":97,"threads_started":1}
W20260811 17:14:41.118600 32553 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:41.112223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:33213 (local address 127.21.54.190:35349)
0811 17:14:41.112425 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:41.112431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:41.112602 (+ 171us) server_negotiation.cc:407] Connection header received
0811 17:14:41.112788 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:41.112793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:41.112853 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:41.112957 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:41.113472 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:41.114082 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:41.114978 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:41.115240 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:41.115368 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:41.118472 (+ 3104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.130:33213: BlockingRecv error: recv got EOF from 127.21.54.130:33213 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 11 17:14:41 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:41 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:41.340387 32555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:41.334213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.131:34813)
0811 17:14:41.334591 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:41.334606 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:41.334754 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:41.335067 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:41.335070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:41.335309 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:41.335408 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:41.335414 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:41.336246 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:41.336252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:41.336995 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:41.337005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:41.337472 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:41.337476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:41.337559 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:41.338157 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:41.338170 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:41.340179 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":117,"threads_started":1}
W20260811 17:14:41.340436 32553 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:41.334311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:34813 (local address 127.21.54.190:35349)
0811 17:14:41.334534 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:41.334540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:41.334689 (+ 149us) server_negotiation.cc:407] Connection header received
0811 17:14:41.334881 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:41.334883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:41.334935 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:41.335031 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:41.335587 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:41.336127 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:41.337129 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:41.337353 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:41.337467 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:41.340325 (+ 2858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.131:34813: BlockingRecv error: recv got EOF from 127.21.54.131:34813 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 11 17:14:41 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:41 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:41.919595 32556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:41.913357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.129:56945)
0811 17:14:41.913696 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:41.913715 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:41.913842 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:41.914167 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:41.914170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:41.914371 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:41.914472 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:41.914478 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:41.915427 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:41.915434 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:41.916177 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:41.916191 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:41.916852 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:41.916855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:41.916924 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:41.917540 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:41.917552 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:41.919470 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":91,"threads_started":1}
W20260811 17:14:41.919852 32557 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:41.913539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:56945 (local address 127.21.54.190:35349)
0811 17:14:41.913870 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:41.913875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:41.913900 (+ 25us) server_negotiation.cc:407] Connection header received
0811 17:14:41.913948 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:41.913950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:41.914004 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:41.914095 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:41.914614 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:41.915289 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:41.916306 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:41.916730 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:41.916852 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:41.919672 (+ 2820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.129:56945: BlockingRecv error: recv got EOF from 127.21.54.129:56945 (error 108)
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":117,"threads_started":1}
Aug 11 17:14:42 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:42 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:42.126645 32558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:42.119759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.130:37007)
0811 17:14:42.120085 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:42.120099 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:42.120213 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:42.120589 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:42.120591 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:42.120784 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:42.120897 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:42.120903 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:42.121974 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:42.121978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:42.122776 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:42.122787 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:42.123377 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:42.123381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:42.123467 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:42.124086 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:42.124098 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:42.126509 (+ 2411us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":116,"threads_started":1}
W20260811 17:14:42.126865 32557 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:42.119939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:37007 (local address 127.21.54.190:35349)
0811 17:14:42.120107 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:42.120111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:42.120178 (+ 67us) server_negotiation.cc:407] Connection header received
0811 17:14:42.120362 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:42.120366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:42.120425 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:42.120554 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:42.121033 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:42.121794 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:42.122905 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:42.123240 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:42.123360 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:42.126718 (+ 3358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.130:37007: BlockingRecv error: recv got EOF from 127.21.54.130:37007 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 17:14:42 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:42 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:42.348948 32560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:42.341609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.131:54203)
0811 17:14:42.341934 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:42.341950 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:42.342112 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:42.342653 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:42.342656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:42.342884 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:42.343166 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:42.343177 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:42.344332 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:42.344338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:42.345198 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:42.345210 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:42.345755 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:42.345758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:42.345828 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:42.346419 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:42.346433 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:42.348747 (+ 2314us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":118,"threads_started":1}
W20260811 17:14:42.349182 32557 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:42.341752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:54203 (local address 127.21.54.190:35349)
0811 17:14:42.341906 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:42.341911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:42.342032 (+ 121us) server_negotiation.cc:407] Connection header received
0811 17:14:42.342277 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:42.342280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:42.342355 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:42.342654 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:42.343369 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:42.344187 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:42.345342 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:42.345610 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:42.345712 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:42.349042 (+ 3330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.131:54203: BlockingRecv error: recv got EOF from 127.21.54.131:54203 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 17:14:42 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:42 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:42.927505 32562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:42.920626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.129:43305)
0811 17:14:42.920993 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:42.921013 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:42.921183 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:42.921638 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:42.921641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:42.921910 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:42.922091 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:42.922103 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:42.923007 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:42.923010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:42.923682 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:42.923692 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:42.924197 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:42.924200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:42.924271 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:42.924910 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:42.924923 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:42.927287 (+ 2364us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":98,"threads_started":1}
W20260811 17:14:42.927589 32563 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:42.920739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:43305 (local address 127.21.54.190:35349)
0811 17:14:42.921065 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:42.921071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:42.921118 (+ 47us) server_negotiation.cc:407] Connection header received
0811 17:14:42.921291 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:42.921294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:42.921370 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:42.921483 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:42.922275 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:42.922875 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:42.923819 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:42.924054 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:42.924154 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:42.927442 (+ 3288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.129:43305: BlockingRecv error: recv got EOF from 127.21.54.129:43305 (error 108)
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":110,"threads_started":1}
Aug 11 17:14:43 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:43 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:43.134075 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:43.127631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.130:41173)
0811 17:14:43.127955 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:43.127969 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:43.128071 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:43.128397 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:43.128400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:43.128622 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:43.128773 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:43.128780 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:43.129742 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:43.129746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:43.130457 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:43.130468 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:43.131074 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:43.131078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:43.131180 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:43.131809 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:43.131823 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:43.133893 (+ 2070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":77,"threads_started":1}
W20260811 17:14:43.134171 32563 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:43.127839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:41173 (local address 127.21.54.190:35349)
0811 17:14:43.128051 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:43.128055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:43.128073 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:14:43.128150 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:43.128153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:43.128232 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:43.128339 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:43.128925 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:43.129608 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:43.130595 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:43.130939 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:43.131068 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:43.134043 (+ 2975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.130:41173: BlockingRecv error: recv got EOF from 127.21.54.130:41173 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 17:14:43 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:43 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:43.356277 32566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:43.349940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.131:43979)
0811 17:14:43.350260 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:43.350276 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:43.350392 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:43.350886 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:43.350889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:43.351113 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:43.351218 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:43.351224 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:43.352137 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:43.352140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:43.352795 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:43.352807 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:43.353389 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:43.353392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:43.353534 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:43.354123 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:43.354135 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:43.356156 (+ 2021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":107,"threads_started":1}
W20260811 17:14:43.356608 32563 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:43.350105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:43979 (local address 127.21.54.190:35349)
0811 17:14:43.350260 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:43.350264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:43.350393 (+ 129us) server_negotiation.cc:407] Connection header received
0811 17:14:43.350484 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:43.350487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:43.350561 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:43.350835 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:43.351354 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:43.351976 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:43.352921 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:43.353241 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:43.353367 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:43.356382 (+ 3015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.131:43979: BlockingRecv error: recv got EOF from 127.21.54.131:43979 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 17:14:43 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:43 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:43.934624 32567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:43.928612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.129:55429)
0811 17:14:43.928908 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:43.928923 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:43.929028 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:43.929389 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:43.929393 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:43.929615 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:43.929710 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:43.929716 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:43.930523 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:43.930528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:43.931103 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:43.931112 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:43.931590 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:43.931594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:43.931682 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:43.932220 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:43.932233 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:43.934506 (+ 2273us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":87,"threads_started":1}
W20260811 17:14:43.934834 32568 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:43.928710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:55429 (local address 127.21.54.190:35349)
0811 17:14:43.928984 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:43.928989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:43.929009 (+ 20us) server_negotiation.cc:407] Connection header received
0811 17:14:43.929150 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:43.929155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:43.929209 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:43.929303 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:43.929843 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:43.930406 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:43.931229 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:43.931472 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:43.931584 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:43.934667 (+ 3083us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.129:55429: BlockingRecv error: recv got EOF from 127.21.54.129:55429 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":88,"threads_started":1}
Aug 11 17:14:44 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:44 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:44.141600 32570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:44.135024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.130:51757)
0811 17:14:44.135347 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:44.135366 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:44.135531 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:44.135878 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:44.135881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:44.136142 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:44.136245 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:44.136252 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:44.137242 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:44.137246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:44.138054 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:44.138065 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:44.138587 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:44.138591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:44.138673 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:44.139320 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:44.139334 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:44.141409 (+ 2075us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":90,"threads_started":1}
W20260811 17:14:44.141777 32568 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:44.135139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:51757 (local address 127.21.54.190:35349)
0811 17:14:44.135330 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:44.135336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:44.135464 (+ 128us) server_negotiation.cc:407] Connection header received
0811 17:14:44.135685 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:44.135695 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:44.135750 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:44.135849 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:44.136386 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:44.137093 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:44.138189 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:44.138450 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:44.138585 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:44.141650 (+ 3065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.130:51757: BlockingRecv error: recv got EOF from 127.21.54.130:51757 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 11 17:14:44 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:44 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:44.363299 32571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:44.357140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.131:46799)
0811 17:14:44.357442 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:44.357456 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:44.357603 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:44.357881 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:44.357883 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:44.358085 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:44.358186 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:44.358192 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:44.358958 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:44.358961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:44.359559 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:44.359569 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:44.360101 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:44.360103 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:44.360169 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:44.360855 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:44.360871 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:44.363098 (+ 2227us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":126,"threads_started":1}
W20260811 17:14:44.363415 32568 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:44.357158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:46799 (local address 127.21.54.190:35349)
0811 17:14:44.357299 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:44.357302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:44.357560 (+ 258us) server_negotiation.cc:407] Connection header received
0811 17:14:44.357713 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:44.357716 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:44.357766 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:44.357837 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:44.358304 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:44.358835 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:44.359698 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:44.359984 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:44.360083 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:44.363271 (+ 3188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.131:46799: BlockingRecv error: recv got EOF from 127.21.54.131:46799 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 17:14:44 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:44 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:44.941951 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:44.935579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.129:51953)
0811 17:14:44.936020 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:44.936038 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:44.936192 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:44.936564 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:44.936570 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:44.936801 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:44.936962 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:44.936970 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:44.938085 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:44.938088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:44.938661 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:44.938671 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:44.939151 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:44.939153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:44.939215 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:44.939815 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:44.939828 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:44.941827 (+ 1999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":118,"threads_started":1}
W20260811 17:14:44.942199 32572 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:44.935655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:51953 (local address 127.21.54.190:35349)
0811 17:14:44.935957 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:44.935962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:44.936127 (+ 165us) server_negotiation.cc:407] Connection header received
0811 17:14:44.936301 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:44.936306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:44.936372 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:44.936474 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:44.937100 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:44.937969 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:44.938782 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:44.939047 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:44.939137 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:44.942013 (+ 2876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.129:51953: BlockingRecv error: recv got EOF from 127.21.54.129:51953 (error 108)
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":114,"threads_started":1}
Aug 11 17:14:45 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:45 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:45.148141 32574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:45.142584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.130:43199)
0811 17:14:45.142924 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:45.142938 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:45.143070 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:45.143319 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:45.143321 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:45.143513 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:45.143611 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:45.143616 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:45.144358 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:45.144360 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:45.144958 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:45.144967 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:45.145404 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:45.145406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:45.145468 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:45.145960 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:45.145972 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:45.147980 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":111,"threads_started":1}
W20260811 17:14:45.148402 32572 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:45.142636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:43199 (local address 127.21.54.190:35349)
0811 17:14:45.142805 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:45.142810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:45.143015 (+ 205us) server_negotiation.cc:407] Connection header received
0811 17:14:45.143160 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:45.143162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:45.143211 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:45.143282 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:45.143720 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:45.144255 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:45.145087 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:45.145303 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:45.145385 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:45.148141 (+ 2756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.130:43199: BlockingRecv error: recv got EOF from 127.21.54.130:43199 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 17:14:45 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:45 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:45.370077 32575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:45.364276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.131:48637)
0811 17:14:45.364686 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:45.364699 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:45.364836 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:45.365155 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:45.365158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:45.365384 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:45.365478 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:45.365483 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:45.366260 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:45.366266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:45.366822 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:45.366830 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:45.367298 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:45.367302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:45.367390 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:45.367977 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:45.367990 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:45.369888 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":117,"threads_started":1}
W20260811 17:14:45.370124 32572 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:45.364354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:48637 (local address 127.21.54.190:35349)
0811 17:14:45.364584 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:45.364589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:45.364784 (+ 195us) server_negotiation.cc:407] Connection header received
0811 17:14:45.364961 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:45.364967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:45.365031 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:45.365130 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:45.365609 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:45.366144 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:45.366946 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:45.367177 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:45.367292 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:45.370018 (+ 2726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.131:48637: BlockingRecv error: recv got EOF from 127.21.54.131:48637 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 17:14:45 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:45 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:45.949266 32577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:45.942991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.129:51177)
0811 17:14:45.943305 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:45.943320 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:45.943441 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:45.943949 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:45.943952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:45.944151 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:45.944254 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:45.944260 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:45.945112 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:45.945115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:45.945682 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:45.945690 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:45.946138 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:45.946141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:45.946206 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:45.946885 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:45.946897 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:45.949143 (+ 2246us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":94,"threads_started":1}
W20260811 17:14:45.949447 32578 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:45.943161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:51177 (local address 127.21.54.190:35349)
0811 17:14:45.943444 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:45.943447 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:45.943465 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:14:45.943576 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:45.943579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:45.943628 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:45.943712 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:45.944395 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:45.944997 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:45.945794 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:45.946019 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:45.946107 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:45.949303 (+ 3196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.129:51177: BlockingRecv error: recv got EOF from 127.21.54.129:51177 (error 108)
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":111,"threads_started":1}
Aug 11 17:14:46 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:46 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:46.155355 32580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:46.149158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.130:60431)
0811 17:14:46.149443 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:46.149458 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:46.149574 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:46.149933 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:46.149935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:46.150130 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:46.150230 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:46.150235 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:46.151243 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:46.151245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:46.151798 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:46.151805 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:46.152281 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:46.152283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:46.152342 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:46.153010 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:46.153031 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:46.155179 (+ 2148us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":108,"threads_started":1}
W20260811 17:14:46.155642 32578 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:46.149215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:60431 (local address 127.21.54.190:35349)
0811 17:14:46.149394 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:46.149399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:46.149563 (+ 164us) server_negotiation.cc:407] Connection header received
0811 17:14:46.149739 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:46.149743 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:46.149798 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:46.149895 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:46.150349 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:46.151132 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:46.151916 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:46.152171 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:46.152262 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:46.155389 (+ 3127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.130:60431: BlockingRecv error: recv got EOF from 127.21.54.130:60431 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 17:14:46 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:46 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:46.377779 32581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:46.371236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.131:38203)
0811 17:14:46.371548 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:46.371572 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:46.371684 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:46.372002 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:46.372004 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:46.372198 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:46.372307 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:46.372313 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:46.373390 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:46.373393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:46.373992 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:46.374012 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:46.374548 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:46.374551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:46.374615 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:46.375243 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:46.375256 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:46.377648 (+ 2392us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":114,"threads_started":1}
W20260811 17:14:46.377990 32578 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:46.371312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:38203 (local address 127.21.54.190:35349)
0811 17:14:46.371499 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:46.371503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:46.371657 (+ 154us) server_negotiation.cc:407] Connection header received
0811 17:14:46.371778 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:46.371781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:46.371834 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:46.371924 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:46.372445 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:46.373260 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:46.374139 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:46.374391 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:46.374488 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:46.377810 (+ 3322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.131:38203: BlockingRecv error: recv got EOF from 127.21.54.131:38203 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 11 17:14:46 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:46 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:46.962441 32583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:46.950129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.129:57735)
0811 17:14:46.950412 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:46.950426 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:46.950524 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:46.950770 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:46.950772 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:46.950961 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:46.951080 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:46.951086 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:46.951981 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:46.951984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:46.952592 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:46.952602 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:46.953156 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:46.953159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:46.953225 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:46.959938 (+ 6713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:46.959954 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:46.962280 (+ 2326us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":100,"threads_started":1}
W20260811 17:14:46.962646 32584 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:46.950248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:57735 (local address 127.21.54.190:35349)
0811 17:14:46.950537 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:46.950540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:46.950556 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:14:46.950596 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:46.950598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:46.950641 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:46.950729 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:46.951236 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:46.951858 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:46.952735 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:46.953011 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:46.953109 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:46.962470 (+ 9361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.129:57735: BlockingRecv error: recv got EOF from 127.21.54.129:57735 (error 108)
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":91,"threads_started":1}
Aug 11 17:14:47 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:47 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:47.162475 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:47.156273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.130:37751)
0811 17:14:47.156649 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:47.156664 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:47.156782 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:47.157055 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:47.157057 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:47.157267 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:47.157371 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:47.157377 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:47.158217 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:47.158220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:47.158855 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:47.158865 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:47.159429 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:47.159433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:47.159532 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:47.160099 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:47.160112 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:47.162281 (+ 2169us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":78,"threads_started":1}
W20260811 17:14:47.162614 32584 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:47.156361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:37751 (local address 127.21.54.190:35349)
0811 17:14:47.156599 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:47.156603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:47.156802 (+ 199us) server_negotiation.cc:407] Connection header received
0811 17:14:47.156860 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:47.156862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:47.156913 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:47.156998 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:47.157500 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:47.158097 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:47.159045 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:47.159279 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:47.159375 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:47.162467 (+ 3092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.130:37751: BlockingRecv error: recv got EOF from 127.21.54.130:37751 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 11 17:14:47 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:47 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:47.385528 32588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:47.378705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.131:46131)
0811 17:14:47.379046 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:47.379062 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:47.379187 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:47.379629 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:47.379632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:47.379858 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:47.379965 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:47.379970 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:47.380805 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:47.380808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:47.381366 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:47.381374 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:47.381883 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:47.381887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:47.381953 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:47.382535 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:47.382553 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:47.385323 (+ 2770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":82,"threads_started":1}
W20260811 17:14:47.385655 32584 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:47.378885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:46131 (local address 127.21.54.190:35349)
0811 17:14:47.379058 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:47.379062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:47.379148 (+ 86us) server_negotiation.cc:407] Connection header received
0811 17:14:47.379338 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:47.379341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:47.379397 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:47.379549 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:47.380106 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:47.380675 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:47.381486 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:47.381765 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:47.381856 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:47.385510 (+ 3654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.131:46131: BlockingRecv error: recv got EOF from 127.21.54.131:46131 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 17:14:47 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:47 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:47.969290 32589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:47.963452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.129:48347)
0811 17:14:47.963795 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:47.963809 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:47.963905 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:47.964204 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:47.964206 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:47.964398 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:47.964500 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:47.964561 (+ 61us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:47.965464 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:47.965467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:47.966050 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:47.966060 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:47.966522 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:47.966524 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:47.966603 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:47.967113 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:47.967124 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:47.969159 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":155,"threads_started":1}
W20260811 17:14:47.969650 32590 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:47.963582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:48347 (local address 127.21.54.190:35349)
0811 17:14:47.963828 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:47.963831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:47.963862 (+ 31us) server_negotiation.cc:407] Connection header received
0811 17:14:47.963994 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:47.963996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:47.964044 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:47.964136 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:47.964749 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:47.965344 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:47.966179 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:47.966412 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:47.966512 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:47.969481 (+ 2969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.129:48347: BlockingRecv error: recv got EOF from 127.21.54.129:48347 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":88,"threads_started":1}
Aug 11 17:14:48 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:48 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:48.170518 32591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:48.163589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.130:51763)
0811 17:14:48.164024 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:48.164046 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:48.164242 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:48.164810 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:48.164814 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:48.165054 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:48.165169 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:48.165177 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:48.166100 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:48.166107 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:48.166747 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:48.166757 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:48.167314 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:48.167319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:48.167434 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:48.168046 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:48.168060 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:48.170304 (+ 2244us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":102,"threads_started":1}
W20260811 17:14:48.170593 32590 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:48.163773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:51763 (local address 127.21.54.190:35349)
0811 17:14:48.164039 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:48.164046 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:48.164167 (+ 121us) server_negotiation.cc:407] Connection header received
0811 17:14:48.164379 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:48.164384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:48.164458 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:48.164600 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:48.165333 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:48.165954 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:48.166912 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:48.167173 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:48.167313 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:48.170467 (+ 3154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.130:51763: BlockingRecv error: recv got EOF from 127.21.54.130:51763 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 11 17:14:48 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:48 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:48.392547 32592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:48.386512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.131:44883)
0811 17:14:48.386838 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:48.386852 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:48.386988 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:48.387324 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:48.387327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:48.387556 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:48.387661 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:48.387667 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:48.388613 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:48.388616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:48.389186 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:48.389194 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:48.389672 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:48.389676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:48.389766 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:48.390271 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:48.390283 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:48.392363 (+ 2080us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":126,"threads_started":1}
W20260811 17:14:48.392858 32590 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:48.386841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:44883 (local address 127.21.54.190:35349)
0811 17:14:48.387010 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:48.387017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:48.387035 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:14:48.387126 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:48.387129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:48.387186 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:48.387300 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:48.387802 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:48.388449 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:48.389313 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:48.389552 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:48.389667 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:48.392603 (+ 2936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.131:44883: BlockingRecv error: recv got EOF from 127.21.54.131:44883 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 17:14:48 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:48 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:48.976819 32593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:48.970296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.129:37165)
0811 17:14:48.970630 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:48.970645 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:48.970742 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:48.971042 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:48.971044 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:48.971235 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:48.971333 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:48.971339 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:48.972237 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:48.972240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:48.972968 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:48.972980 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:48.973565 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:48.973568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:48.973638 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:48.974263 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:48.974276 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:48.976686 (+ 2410us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":108,"threads_started":1}
W20260811 17:14:48.977072 32594 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:48.970361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:37165 (local address 127.21.54.190:35349)
0811 17:14:48.970686 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:48.970690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:48.970705 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:14:48.970839 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:48.970841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:48.970889 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:48.971006 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:48.971469 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:48.972082 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:48.973120 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:48.973366 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:48.973464 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:48.976847 (+ 3383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.129:37165: BlockingRecv error: recv got EOF from 127.21.54.129:37165 (error 108)
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":159,"threads_started":1}
Aug 11 17:14:49 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:49 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:49.178258 32595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:49.171688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.130:48453)
0811 17:14:49.172064 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:49.172080 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:49.172221 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:49.172623 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:49.172626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:49.172828 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:49.172936 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:49.172941 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:49.173876 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:49.173884 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:49.174563 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:49.174574 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:49.175177 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:49.175180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:49.175291 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:49.176050 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:49.176068 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:49.178127 (+ 2059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":106,"threads_started":1}
W20260811 17:14:49.178448 32594 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:49.171820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:48453 (local address 127.21.54.190:35349)
0811 17:14:49.172089 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:49.172092 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:49.172206 (+ 114us) server_negotiation.cc:407] Connection header received
0811 17:14:49.172315 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:49.172317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:49.172370 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:49.172456 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:49.173116 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:49.173751 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:49.174714 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:49.175039 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:49.175157 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:49.178284 (+ 3127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.130:48453: BlockingRecv error: recv got EOF from 127.21.54.130:48453 (error 108)
Metrics: {"server-negotiator.queue_time_us":157}
Aug 11 17:14:49 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:49 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:49.400444 32596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:49.393486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.131:32995)
0811 17:14:49.393797 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:49.393813 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:49.393934 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:49.394277 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:49.394280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:49.394566 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:49.394688 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:49.394694 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:49.395556 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:49.395559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:49.396177 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:49.396187 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:49.396864 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:49.396868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:49.396954 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:49.397680 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:49.397729 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:49.400262 (+ 2533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":125,"threads_started":1}
W20260811 17:14:49.400792 32594 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:49.393732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:32995 (local address 127.21.54.190:35349)
0811 17:14:49.393879 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:49.393883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:49.393900 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:14:49.394049 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:49.394052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:49.394106 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:49.394201 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:49.394849 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:49.395398 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:49.396333 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:49.396713 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:49.396817 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:49.400412 (+ 3595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.131:32995: BlockingRecv error: recv got EOF from 127.21.54.131:32995 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 17:14:49 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:49 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:49.984239 32597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:49.977881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.129:54003)
0811 17:14:49.978209 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:49.978225 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:49.978341 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:49.978633 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:49.978636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:49.978844 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:49.978979 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:49.978986 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:49.979870 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:49.979874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:49.980471 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:49.980481 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:49.981143 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:49.981147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:49.981224 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:49.981805 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:49.981817 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:49.984111 (+ 2294us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":119,"threads_started":1}
W20260811 17:14:49.984408 32598 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:49.978057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:54003 (local address 127.21.54.190:35349)
0811 17:14:49.978332 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:49.978336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:49.978353 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:14:49.978423 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:49.978425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:49.978474 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:49.978575 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:49.979128 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:49.979727 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:49.980676 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:49.980941 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:49.981078 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:49.984257 (+ 3179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.129:54003: BlockingRecv error: recv got EOF from 127.21.54.129:54003 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":105,"threads_started":1}
Aug 11 17:14:50 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:50 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:50.186169 32599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:50.179302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.130:38857)
0811 17:14:50.179733 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:50.179747 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:50.179893 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:50.180222 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:50.180225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:50.180470 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:50.180630 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:50.180637 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:50.181540 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:50.181545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:50.182197 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:50.182208 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:50.182767 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:50.182771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:50.182885 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:50.183556 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:50.183574 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:50.186026 (+ 2452us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":104,"threads_started":1}
W20260811 17:14:50.186581 32598 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:50.179455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:38857 (local address 127.21.54.190:35349)
0811 17:14:50.179652 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:50.179658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:50.179834 (+ 176us) server_negotiation.cc:407] Connection header received
0811 17:14:50.180024 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:50.180027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:50.180091 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:50.180192 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:50.180781 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:50.181392 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:50.182346 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:50.182629 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:50.182760 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:50.186213 (+ 3453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.130:38857: BlockingRecv error: recv got EOF from 127.21.54.130:38857 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 17:14:50 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:50 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:50.407364 32600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:50.401390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.131:49161)
0811 17:14:50.401714 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:50.401728 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:50.401826 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:50.402085 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:50.402087 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:50.402296 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:50.402398 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:50.402403 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:50.403328 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:50.403331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:50.403955 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:50.403964 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:50.404458 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:50.404461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:50.404557 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:50.405155 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:50.405168 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:50.407231 (+ 2063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":102,"threads_started":1}
W20260811 17:14:50.407559 32598 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:50.401600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:49161 (local address 127.21.54.190:35349)
0811 17:14:50.401765 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:50.401769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:50.401785 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:14:50.401916 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:50.401918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:50.401967 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:50.402045 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:50.402595 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:50.403194 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:50.404095 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:50.404342 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:50.404434 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:50.407402 (+ 2968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.131:49161: BlockingRecv error: recv got EOF from 127.21.54.131:49161 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 11 17:14:50 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:50 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:50.991283 32601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:50.985251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.129:34307)
0811 17:14:50.985597 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:50.985617 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:50.985736 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:50.986033 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:50.986036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:50.986288 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:50.986429 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:50.986435 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:50.987363 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:50.987366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:50.987956 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:50.987965 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:50.988461 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:50.988464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:50.988645 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:50.989215 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:50.989226 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:50.991124 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":100,"threads_started":1}
W20260811 17:14:50.991425 32604 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:50.985409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:34307 (local address 127.21.54.190:35349)
0811 17:14:50.985707 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:50.985712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:50.985728 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:14:50.985831 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:50.985834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:50.985889 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:50.985991 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:50.986581 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:50.987246 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:50.988094 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:50.988349 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:50.988447 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:50.991280 (+ 2833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.129:34307: BlockingRecv error: recv got EOF from 127.21.54.129:34307 (error 108)
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":110,"threads_started":1}
Aug 11 17:14:51 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:51 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:51.193197 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:51.187084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.130:43165)
0811 17:14:51.187376 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:51.187389 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:51.187487 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:51.187729 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:51.187731 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:51.187921 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:51.188043 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:51.188048 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:51.188986 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:51.188989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:51.189568 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:51.189579 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:51.190028 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:51.190031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:51.190111 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:51.190627 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:51.190639 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:51.193012 (+ 2373us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":74,"threads_started":1}
W20260811 17:14:51.193271 32604 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:51.187332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:43165 (local address 127.21.54.190:35349)
0811 17:14:51.187506 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:51.187509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:51.187525 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:14:51.187565 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:51.187567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:51.187610 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:51.187686 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:51.188179 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:51.188858 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:51.189688 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:51.189909 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:51.189998 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:51.193150 (+ 3152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.130:43165: BlockingRecv error: recv got EOF from 127.21.54.130:43165 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 11 17:14:51 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:51 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:51.414880 32606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:51.408602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.131:42997)
0811 17:14:51.408969 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:51.408987 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:51.409164 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:51.409447 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:51.409450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:51.409686 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:51.409782 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:51.409788 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:51.410760 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:51.410766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:51.411596 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:51.411606 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:51.412079 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:51.412082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:51.412160 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:51.412779 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:51.412793 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:51.414740 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":94,"threads_started":1}
W20260811 17:14:51.414999 32604 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:51.408760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:42997 (local address 127.21.54.190:35349)
0811 17:14:51.408953 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:51.408958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:51.409096 (+ 138us) server_negotiation.cc:407] Connection header received
0811 17:14:51.409273 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:51.409276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:51.409336 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:51.409429 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:51.409913 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:51.410603 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:51.411708 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:51.411934 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:51.412019 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:51.414878 (+ 2859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.131:42997: BlockingRecv error: recv got EOF from 127.21.54.131:42997 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 17:14:52 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:52 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:52.005016 32607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:51.992237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.129:33399)
0811 17:14:51.992696 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:51.992710 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:51.992850 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:51.993179 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:51.993181 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:51.993373 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:51.993522 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:51.993528 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:51.994406 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:51.994409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:51.995087 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:51.995098 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:51.995668 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:51.995672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:51.995750 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:52.002445 (+ 6695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:52.002460 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:52.004862 (+ 2402us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":102,"threads_started":1}
W20260811 17:14:52.005259 32609 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:51.992362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:33399 (local address 127.21.54.190:35349)
0811 17:14:51.992781 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:51.992786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:51.992804 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:14:51.992959 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:51.992962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:51.993040 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:51.993134 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:51.993637 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:51.994261 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:51.995242 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:51.995524 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:51.995626 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:52.005062 (+ 9436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.129:33399: BlockingRecv error: recv got EOF from 127.21.54.129:33399 (error 108)
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":112,"threads_started":1}
Aug 11 17:14:52 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:52 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:52.200834 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:52.194192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.130:35839)
0811 17:14:52.194550 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:52.194570 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:52.194749 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:52.195109 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:52.195112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:52.195360 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:52.195467 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:52.195475 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:52.196368 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:52.196374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:52.197070 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:52.197079 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:52.197639 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:52.197642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:52.197737 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:52.198364 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:52.198376 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:52.200608 (+ 2232us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":116,"threads_started":1}
W20260811 17:14:52.200888 32609 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:52.194267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:35839 (local address 127.21.54.190:35349)
0811 17:14:52.194532 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:52.194537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:52.194676 (+ 139us) server_negotiation.cc:407] Connection header received
0811 17:14:52.194899 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:52.194903 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:52.194973 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:52.195076 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:52.195619 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:52.196227 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:52.197222 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:52.197494 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:52.197633 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:52.200758 (+ 3125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.130:35839: BlockingRecv error: recv got EOF from 127.21.54.130:35839 (error 108)
Metrics: {"server-negotiator.queue_time_us":132}
Aug 11 17:14:52 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:52 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.131@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:52.422688 32612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:52.415816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.131:51359)
0811 17:14:52.416242 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:52.416257 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:52.416419 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:52.416819 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:52.416822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:52.417098 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:52.417206 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:52.417212 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:52.418072 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:52.418079 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:52.418720 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:52.418730 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:52.419357 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:52.419362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:52.419457 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:52.420188 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:52.420202 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:52.422540 (+ 2338us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":134,"threads_started":1}
W20260811 17:14:52.422829 32609 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:52.415969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:51359 (local address 127.21.54.190:35349)
0811 17:14:52.416172 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:52.416178 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:52.416349 (+ 171us) server_negotiation.cc:407] Connection header received
0811 17:14:52.416622 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:52.416625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:52.416680 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:52.416785 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:52.417346 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:52.417943 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:52.418871 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:52.419178 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:52.419303 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:52.422701 (+ 3398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.131:51359: BlockingRecv error: recv got EOF from 127.21.54.131:51359 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 17:14:53 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:53 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.129@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:53.012357 32613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:53.006112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.129:48613)
0811 17:14:53.006407 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:53.006422 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:53.006527 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:53.006886 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:53.006888 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:53.007121 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:53.007226 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:53.007233 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:53.008044 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:53.008086 (+ 42us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:53.008885 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:53.008896 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:53.009433 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:53.009436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:53.009503 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:53.010107 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:53.010119 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:53.012235 (+ 2116us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":108,"threads_started":1}
W20260811 17:14:53.012616 32614 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:53.006204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:48613 (local address 127.21.54.190:35349)
0811 17:14:53.006471 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:53.006475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:53.006489 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:14:53.006667 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:53.006670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:53.006727 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:53.006847 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:53.007364 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:53.007946 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:53.009037 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:53.009301 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:53.009470 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:53.012426 (+ 2956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.129:48613: BlockingRecv error: recv got EOF from 127.21.54.129:48613 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":100,"threads_started":1}
Aug 11 17:14:53 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:14:53 dist-test-slave-fd7l krb5kdc[31347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.54.130@KRBTEST.COM for oryx/127.21.54.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:14:53.207688 32615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:53.202021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35349 (local address 127.21.54.130:59957)
0811 17:14:53.202327 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:53.202340 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:53.202436 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:53.202704 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:53.202706 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:53.202909 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:53.203033 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:53.203039 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:53.203830 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:53.203833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:53.204477 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:53.204487 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:53.205010 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:53.205012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:53.205099 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:53.205621 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:53.205633 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:53.207562 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.54.190:35349: Server oryx/127.21.54.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":131,"threads_started":1}
W20260811 17:14:53.207852 32614 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:53.202188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:59957 (local address 127.21.54.190:35349)
0811 17:14:53.202343 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:53.202348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:53.202394 (+ 46us) server_negotiation.cc:407] Connection header received
0811 17:14:53.202525 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:53.202528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:53.202582 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:53.202664 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:53.203145 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:53.203700 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:53.204685 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:53.204905 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:53.204990 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:53.207721 (+ 2731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.54.130:59957: BlockingRecv error: recv got EOF from 127.21.54.130:59957 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:14:53.294600 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 32042
I20260811 17:14:53.304878 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 32221
I20260811 17:14:53.313841 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 32377
I20260811 17:14:53.322008 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 31960
2026-08-11T17:14:53Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18359 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T17:14:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:14:53Z Disabled control of system clock
I20260811 17:14:53.358711 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:46031
--webserver_interface=127.21.54.190
--webserver_port=0
--builtin_ntp_servers=127.21.54.148:42649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:46031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 17:14:53.483824 32619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:53.484098 32619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:53.484134 32619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:53.488214 32619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:53.488281 32619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:53.488297 32619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:53.488314 32619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:53.488329 32619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:14:53.493368 32619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:42649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:46031
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:46031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:53.494540 32619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:53.495995 32619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:14:53.505412 32619 server_base.cc:1034] running on GCE node
W20260811 17:14:53.505360 32628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:53.505327 32625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:53.505347 32626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:53.506278 32619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:53.507481 32619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:53.508658 32619 hybrid_clock.cc:648] HybridClock initialized: now 1786468493508644 us; error 26 us; skew 500 ppm
I20260811 17:14:53.510689 32619 webserver.cc:460] Webserver started at http://127.21.54.190:40207/ using document root <none> and password file <none>
I20260811 17:14:53.511318 32619 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:53.511427 32619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:53.511662 32619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:53.513854 32619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "cbabad5d13304ed481ee6990eb03f424"
format_stamp: "Formatted at 2026-08-11 17:14:53 on dist-test-slave-fd7l"
server_key: "75e63f26a34b4eed44ae98175ce13b5e"
server_key_iv: "aaa30a71f760af9fd912acee9930954b"
server_key_version: "encryptionkey@0"
I20260811 17:14:53.514456 32619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "cbabad5d13304ed481ee6990eb03f424"
format_stamp: "Formatted at 2026-08-11 17:14:53 on dist-test-slave-fd7l"
server_key: "75e63f26a34b4eed44ae98175ce13b5e"
server_key_iv: "aaa30a71f760af9fd912acee9930954b"
server_key_version: "encryptionkey@0"
I20260811 17:14:53.518407 32619 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:14:53.520989 32635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:53.522554 32619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:53.522727 32619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "cbabad5d13304ed481ee6990eb03f424"
format_stamp: "Formatted at 2026-08-11 17:14:53 on dist-test-slave-fd7l"
server_key: "75e63f26a34b4eed44ae98175ce13b5e"
server_key_iv: "aaa30a71f760af9fd912acee9930954b"
server_key_version: "encryptionkey@0"
I20260811 17:14:53.522863 32619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:53.545266 32619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:53.546078 32619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:53.546309 32619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:53.553653 32619 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:46031
I20260811 17:14:53.553704 32701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:46031 every 8 connection(s)
I20260811 17:14:53.554661 32619 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:14:53.556270 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 32619
I20260811 17:14:53.556396 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:14:53.556746 21722 external_mini_cluster.cc:1428] Setting key 5fcc150c896164c76e84b23d76cb1174
I20260811 17:14:53.558312 32702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:53.564121 32702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424: Bootstrap starting.
I20260811 17:14:53.566236 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:53.558290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59350 (local address 127.21.54.190:46031)
0811 17:14:53.558956 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:53.558976 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:53.559012 (+ 36us) server_negotiation.cc:407] Connection header received
0811 17:14:53.559704 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:53.559721 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:53.560101 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:53.560437 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:53.560895 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:53.561860 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:53.562762 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:53.563235 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:53.563407 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:53.564401 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:53.564406 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:53.564418 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:53.564925 (+ 507us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:53.564982 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:53.565003 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:53.565112 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:53.565274 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 17:14:53.565536 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":159,"threads_started":1}
I20260811 17:14:53.566499 32702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:53.567257 32702 log.cc:824] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:53.569320 32702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424: No bootstrap required, opened a new log
I20260811 17:14:53.572961 32702 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbabad5d13304ed481ee6990eb03f424" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46031 } }
I20260811 17:14:53.573231 32702 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:53.573316 32702 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbabad5d13304ed481ee6990eb03f424, State: Initialized, Role: FOLLOWER
I20260811 17:14:53.573755 32702 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbabad5d13304ed481ee6990eb03f424" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46031 } }
I20260811 17:14:53.573912 32702 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:14:53.574005 32702 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:14:53.574131 32702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:53.575136 32702 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbabad5d13304ed481ee6990eb03f424" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46031 } }
I20260811 17:14:53.575518 32702 leader_election.cc:302] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbabad5d13304ed481ee6990eb03f424; no voters:
I20260811 17:14:53.575934 32702 leader_election.cc:288] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:14:53.576023 32707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:53.576247 32707 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424 [term 1 LEADER]: Becoming Leader. State: Replica: cbabad5d13304ed481ee6990eb03f424, State: Running, Role: LEADER
I20260811 17:14:53.576624 32707 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbabad5d13304ed481ee6990eb03f424" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46031 } }
I20260811 17:14:53.577359 32702 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:53.578500 32710 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbabad5d13304ed481ee6990eb03f424" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbabad5d13304ed481ee6990eb03f424" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46031 } } }
I20260811 17:14:53.578644 32710 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:53.578989 32708 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbabad5d13304ed481ee6990eb03f424. Latest consensus state: current_term: 1 leader_uuid: "cbabad5d13304ed481ee6990eb03f424" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbabad5d13304ed481ee6990eb03f424" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46031 } } }
I20260811 17:14:53.579068 32718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:14:53.579150 32708 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:53.582784 32718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:14:53.589501 32718 catalog_manager.cc:1347] Generated new cluster ID: 28643ff3c6bf48c8b1bc74bde4fc3b36
I20260811 17:14:53.589593 32718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:14:53.606134 32718 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:14:53.607543 32718 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:14:53.615525 32718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbabad5d13304ed481ee6990eb03f424: Generated new TSK 0
I20260811 17:14:53.616449 32718 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:14:53.622715 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:46031
--builtin_ntp_servers=127.21.54.148:42649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 17:14:53.739938 32731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:53.740244 32731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:53.740324 32731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:53.744436 32731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:53.744566 32731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:53.744699 32731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:14:53.749809 32731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:42649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:46031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:53.750973 32731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:53.752488 32731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:53.762522 32737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:53.762653 32731 server_base.cc:1034] running on GCE node
W20260811 17:14:53.762522 32740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:53.762586 32738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:53.763317 32731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:53.764003 32731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:53.765216 32731 hybrid_clock.cc:648] HybridClock initialized: now 1786468493765200 us; error 32 us; skew 500 ppm
I20260811 17:14:53.767232 32731 webserver.cc:460] Webserver started at http://127.21.54.129:40711/ using document root <none> and password file <none>
I20260811 17:14:53.767879 32731 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:53.767980 32731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:53.768231 32731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:53.770485 32731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "f7ebe54f6b8b4bebb32f67bb9e675a1c"
format_stamp: "Formatted at 2026-08-11 17:14:53 on dist-test-slave-fd7l"
server_key: "49b2bea476937332f181daf2b0611dcf"
server_key_iv: "e1a78d75220d46b8deefc179a7a1286c"
server_key_version: "encryptionkey@0"
I20260811 17:14:53.771064 32731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "f7ebe54f6b8b4bebb32f67bb9e675a1c"
format_stamp: "Formatted at 2026-08-11 17:14:53 on dist-test-slave-fd7l"
server_key: "49b2bea476937332f181daf2b0611dcf"
server_key_iv: "e1a78d75220d46b8deefc179a7a1286c"
server_key_version: "encryptionkey@0"
I20260811 17:14:53.774994 32731 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:14:53.777869 32747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:53.779278 32731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:14:53.779423 32731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "f7ebe54f6b8b4bebb32f67bb9e675a1c"
format_stamp: "Formatted at 2026-08-11 17:14:53 on dist-test-slave-fd7l"
server_key: "49b2bea476937332f181daf2b0611dcf"
server_key_iv: "e1a78d75220d46b8deefc179a7a1286c"
server_key_version: "encryptionkey@0"
I20260811 17:14:53.779598 32731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:53.795884 32731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:53.796733 32731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:53.796945 32731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:53.797574 32731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:53.798820 32731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:53.798894 32731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:53.798962 32731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:53.799021 32731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:53.809271 32731 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:37067
I20260811 17:14:53.809291 428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:37067 every 8 connection(s)
I20260811 17:14:53.810293 32731 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:14:53.810678 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 32731
I20260811 17:14:53.810806 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:14:53.811175 21722 external_mini_cluster.cc:1428] Setting key 6398948e5cb95918dbabf0d89a4b37e5
I20260811 17:14:53.813555 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:46031
--builtin_ntp_servers=127.21.54.148:42649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 17:14:53.822515 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:53.812482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:40771 (local address 127.21.54.190:46031)
0811 17:14:53.812653 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:53.812660 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:53.814022 (+ 1362us) server_negotiation.cc:407] Connection header received
0811 17:14:53.816056 (+ 2034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:53.816063 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:53.816133 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:53.816236 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:53.817477 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:53.818053 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:53.819999 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:53.820244 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:53.820333 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:53.821967 (+ 1634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:53.821970 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:53.821972 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:53.821986 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:53.822008 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:53.822024 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:53.822088 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:53.822317 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 17:14:53.822388 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 17:14:53.823864 432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:53.813013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:46031 (local address 127.21.54.129:40771)
0811 17:14:53.813835 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:53.813895 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:53.815819 (+ 1924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:53.816404 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:53.816410 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:53.816495 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:53.817287 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:53.817312 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:53.818204 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:53.818211 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:53.819850 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:53.819865 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:53.820424 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:53.820431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:53.820725 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:53.821612 (+ 887us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:14:53.821617 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:53.821683 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:14:53.821785 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:53.822125 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:53.822149 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:53.822158 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 17:14:53.822306 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 17:14:53.822657 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":115,"threads_started":1}
I20260811 17:14:53.825374 429 heartbeater.cc:347] Connected to a master server at 127.21.54.190:46031
I20260811 17:14:53.825750 429 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:53.826366 429 heartbeater.cc:511] Master 127.21.54.190:46031 requested a full tablet report, sending...
I20260811 17:14:53.828333 32657 ts_manager.cc:194] Registered new tserver with Master: f7ebe54f6b8b4bebb32f67bb9e675a1c (127.21.54.129:37067)
I20260811 17:14:53.829668 32657 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.54.129:40771
W20260811 17:14:53.942113 433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:53.942443 433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:53.942524 433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:53.946708 433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:53.946787 433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:53.946870 433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:14:53.952229 433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:42649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:46031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:53.953462 433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:53.954936 433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:53.965018 438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:53.965025 439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:53.965015 441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:53.965368 433 server_base.cc:1034] running on GCE node
I20260811 17:14:53.965785 433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:53.966441 433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:53.967617 433 hybrid_clock.cc:648] HybridClock initialized: now 1786468493967598 us; error 28 us; skew 500 ppm
I20260811 17:14:53.969817 433 webserver.cc:460] Webserver started at http://127.21.54.130:38625/ using document root <none> and password file <none>
I20260811 17:14:53.970469 433 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:53.970541 433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:53.970827 433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:53.973318 433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "7450f6784b144b24ade2f848d9a6933e"
format_stamp: "Formatted at 2026-08-11 17:14:53 on dist-test-slave-fd7l"
server_key: "b808982dd6e100ca3787fa7db7f2eda5"
server_key_iv: "04c5d44184a1a76d7bfd59c830e4c17a"
server_key_version: "encryptionkey@0"
I20260811 17:14:53.973943 433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "7450f6784b144b24ade2f848d9a6933e"
format_stamp: "Formatted at 2026-08-11 17:14:53 on dist-test-slave-fd7l"
server_key: "b808982dd6e100ca3787fa7db7f2eda5"
server_key_iv: "04c5d44184a1a76d7bfd59c830e4c17a"
server_key_version: "encryptionkey@0"
I20260811 17:14:53.978183 433 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:14:53.981287 448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:53.982841 433 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 17:14:53.982980 433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "7450f6784b144b24ade2f848d9a6933e"
format_stamp: "Formatted at 2026-08-11 17:14:53 on dist-test-slave-fd7l"
server_key: "b808982dd6e100ca3787fa7db7f2eda5"
server_key_iv: "04c5d44184a1a76d7bfd59c830e4c17a"
server_key_version: "encryptionkey@0"
I20260811 17:14:53.983112 433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:54.005172 433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:54.005982 433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:54.006215 433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:54.006913 433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:54.008108 433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:54.008183 433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:54.008257 433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:54.008301 433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:54.018015 433 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:33473
I20260811 17:14:54.018042 582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:33473 every 8 connection(s)
I20260811 17:14:54.018991 433 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:14:54.022480 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 433
I20260811 17:14:54.022624 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:14:54.022934 21722 external_mini_cluster.cc:1428] Setting key 9222b207fccb2ae01dadd0579dd8c78f
I20260811 17:14:54.025241 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:46031
--builtin_ntp_servers=127.21.54.148:42649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 17:14:54.030088 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:54.020838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:50433 (local address 127.21.54.190:46031)
0811 17:14:54.021003 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:54.021007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:54.022327 (+ 1320us) server_negotiation.cc:407] Connection header received
0811 17:14:54.024100 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:54.024103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:54.024153 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:54.024233 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:54.025310 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:54.025894 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:54.027603 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:54.027950 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:54.028068 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:54.029512 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:54.029515 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:54.029517 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:54.029532 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:54.029554 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:54.029563 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:54.029644 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:54.029880 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 17:14:54.029947 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 17:14:54.031394 586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:54.021425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:46031 (local address 127.21.54.130:50433)
0811 17:14:54.022152 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:54.022195 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:54.023829 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:54.024455 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:54.024464 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:54.024616 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:54.025154 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:54.025167 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:54.026036 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:54.026039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:54.027366 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:54.027383 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:54.028136 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:54.028143 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:54.028379 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:54.029155 (+ 776us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:14:54.029159 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:54.029219 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:14:54.029312 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:54.029665 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:54.029676 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:54.029686 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 17:14:54.029838 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 17:14:54.030197 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":116,"threads_started":1}
I20260811 17:14:54.032727 583 heartbeater.cc:347] Connected to a master server at 127.21.54.190:46031
I20260811 17:14:54.033018 583 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:54.033581 583 heartbeater.cc:511] Master 127.21.54.190:46031 requested a full tablet report, sending...
I20260811 17:14:54.034726 32657 ts_manager.cc:194] Registered new tserver with Master: 7450f6784b144b24ade2f848d9a6933e (127.21.54.130:33473)
I20260811 17:14:54.035322 32657 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.54.130:50433
W20260811 17:14:54.144538 587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:54.144819 587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:54.144858 587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:54.149037 587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:54.149106 587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:54.149189 587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:14:54.154078 587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:42649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:46031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:54.155279 587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:54.156762 587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:54.167474 593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:54.167486 596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:54.167481 592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:54.167833 587 server_base.cc:1034] running on GCE node
I20260811 17:14:54.168363 587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:54.169122 587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:54.170497 587 hybrid_clock.cc:648] HybridClock initialized: now 1786468494170311 us; error 36 us; skew 500 ppm
I20260811 17:14:54.172575 587 webserver.cc:460] Webserver started at http://127.21.54.131:37699/ using document root <none> and password file <none>
I20260811 17:14:54.173192 587 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:54.173290 587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:54.173547 587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:54.175720 587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "4733c8ee82484831b38b6f374d93e70e"
format_stamp: "Formatted at 2026-08-11 17:14:54 on dist-test-slave-fd7l"
server_key: "fd82242bb7d4466da79cc1c91d4948a2"
server_key_iv: "0372cce10566c7e3213c1b5e2cf0bf1d"
server_key_version: "encryptionkey@0"
I20260811 17:14:54.176316 587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "4733c8ee82484831b38b6f374d93e70e"
format_stamp: "Formatted at 2026-08-11 17:14:54 on dist-test-slave-fd7l"
server_key: "fd82242bb7d4466da79cc1c91d4948a2"
server_key_iv: "0372cce10566c7e3213c1b5e2cf0bf1d"
server_key_version: "encryptionkey@0"
I20260811 17:14:54.180323 587 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260811 17:14:54.183660 602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:54.185107 587 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:54.185240 587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "4733c8ee82484831b38b6f374d93e70e"
format_stamp: "Formatted at 2026-08-11 17:14:54 on dist-test-slave-fd7l"
server_key: "fd82242bb7d4466da79cc1c91d4948a2"
server_key_iv: "0372cce10566c7e3213c1b5e2cf0bf1d"
server_key_version: "encryptionkey@0"
I20260811 17:14:54.185374 587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:54.209625 587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:54.210440 587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:54.210683 587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:54.211340 587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:54.212392 587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:54.212473 587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:54.212589 587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:54.212641 587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:54.222077 587 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:43079
I20260811 17:14:54.222105 740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:43079 every 8 connection(s)
I20260811 17:14:54.223096 587 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:14:54.223819 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 587
I20260811 17:14:54.223975 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:14:54.224323 21722 external_mini_cluster.cc:1428] Setting key d7a80e019dfe6c478db6ebe337636288
I20260811 17:14:54.233544 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:54.225506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:36553 (local address 127.21.54.190:46031)
0811 17:14:54.225671 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:54.225676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:54.227000 (+ 1324us) server_negotiation.cc:407] Connection header received
0811 17:14:54.228616 (+ 1616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:54.228618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:54.228665 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:54.228734 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:54.229659 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:54.230174 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:54.231557 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:54.231769 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:54.231851 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:54.233070 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:54.233073 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:54.233074 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:54.233086 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:54.233107 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:54.233114 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:54.233174 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:54.233333 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 17:14:54.233384 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 17:14:54.234529 745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:54.225901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:46031 (local address 127.21.54.131:36553)
0811 17:14:54.226804 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:54.226851 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:54.228350 (+ 1499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:54.228887 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:54.228894 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:54.228983 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:54.229524 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:54.229537 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:54.230327 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:54.230330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:54.231366 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:54.231378 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:54.231919 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:54.231923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:54.232095 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:54.232785 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:14:54.232787 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:54.232831 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:14:54.232906 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:54.233197 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:54.233204 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:54.233209 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 17:14:54.233322 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 17:14:54.233621 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":117,"threads_started":1}
I20260811 17:14:54.235734 742 heartbeater.cc:347] Connected to a master server at 127.21.54.190:46031
I20260811 17:14:54.235954 742 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:54.236465 742 heartbeater.cc:511] Master 127.21.54.190:46031 requested a full tablet report, sending...
I20260811 17:14:54.237628 32657 ts_manager.cc:194] Registered new tserver with Master: 4733c8ee82484831b38b6f374d93e70e (127.21.54.131:43079)
I20260811 17:14:54.238513 32657 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.54.131:36553
I20260811 17:14:54.238802 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 17:14:54.241883 32705 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:54.240878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59354 (local address 127.21.54.190:46031)
0811 17:14:54.241024 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:54.241029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:54.241258 (+ 229us) server_negotiation.cc:407] Connection header received
0811 17:14:54.241372 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:54.241374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:54.241414 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:54.241486 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:54.241753 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59354: BlockingRecv error: recv got EOF from 127.0.0.1:59354 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 17:14:54.242015 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 32731
I20260811 17:14:54.252403 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 433
I20260811 17:14:54.261456 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 587
I20260811 17:14:54.272759 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 32619
2026-08-11T17:14:54Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (950 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T17:14:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:14:54Z Disabled control of system clock
I20260811 17:14:54.307533 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:35269
--webserver_interface=127.21.54.190
--webserver_port=0
--builtin_ntp_servers=127.21.54.148:46711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:35269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 17:14:54.426244 755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:54.426540 755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:54.426581 755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:54.430642 755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:54.430709 755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:54.430725 755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:54.430742 755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:54.430757 755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:14:54.435803 755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:46711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:35269
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:35269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:54.437032 755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:54.438414 755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:54.447537 761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:54.447537 762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:54.447711 755 server_base.cc:1034] running on GCE node
W20260811 17:14:54.447919 766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:54.448556 755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:54.449838 755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:54.451013 755 hybrid_clock.cc:648] HybridClock initialized: now 1786468494450983 us; error 44 us; skew 500 ppm
I20260811 17:14:54.453142 755 webserver.cc:460] Webserver started at http://127.21.54.190:35665/ using document root <none> and password file <none>
I20260811 17:14:54.453799 755 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:54.453890 755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:54.454133 755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:54.456300 755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "4e459b08a9904961afb97b31bc4778d0"
format_stamp: "Formatted at 2026-08-11 17:14:54 on dist-test-slave-fd7l"
server_key: "d0985c2c04d7ffea6e7cae8b18925a70"
server_key_iv: "e6eee1955ca1b1bb02bfaf5ed294a6d5"
server_key_version: "encryptionkey@0"
I20260811 17:14:54.456959 755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "4e459b08a9904961afb97b31bc4778d0"
format_stamp: "Formatted at 2026-08-11 17:14:54 on dist-test-slave-fd7l"
server_key: "d0985c2c04d7ffea6e7cae8b18925a70"
server_key_iv: "e6eee1955ca1b1bb02bfaf5ed294a6d5"
server_key_version: "encryptionkey@0"
I20260811 17:14:54.460879 755 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:14:54.463498 773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:54.464946 755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:54.465094 755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "4e459b08a9904961afb97b31bc4778d0"
format_stamp: "Formatted at 2026-08-11 17:14:54 on dist-test-slave-fd7l"
server_key: "d0985c2c04d7ffea6e7cae8b18925a70"
server_key_iv: "e6eee1955ca1b1bb02bfaf5ed294a6d5"
server_key_version: "encryptionkey@0"
I20260811 17:14:54.465221 755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:54.470085 755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:54.470734 755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:54.470932 755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:54.478410 755 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:35269
I20260811 17:14:54.478430 832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:35269 every 8 connection(s)
I20260811 17:14:54.479400 755 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:14:54.482954 835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:54.484493 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 755
I20260811 17:14:54.484699 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:14:54.485002 21722 external_mini_cluster.cc:1428] Setting key fab276062efdd5c0445684a132b8705a
I20260811 17:14:54.489962 835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0: Bootstrap starting.
I20260811 17:14:54.490953 839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:54.486577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60632 (local address 127.21.54.190:35269)
0811 17:14:54.487238 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:54.487256 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:54.487292 (+ 36us) server_negotiation.cc:407] Connection header received
0811 17:14:54.488015 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:54.488036 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:54.488388 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:54.488739 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:54.489655 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:54.489660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:54.489671 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:54.489681 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:54.489737 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:54.489753 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:54.489866 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:54.489987 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 17:14:54.490263 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":141,"threads_started":1}
I20260811 17:14:54.492601 835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:54.493464 835 log.cc:824] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:54.495509 835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0: No bootstrap required, opened a new log
I20260811 17:14:54.499221 835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e459b08a9904961afb97b31bc4778d0" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35269 } }
I20260811 17:14:54.499526 835 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:54.499605 835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e459b08a9904961afb97b31bc4778d0, State: Initialized, Role: FOLLOWER
I20260811 17:14:54.500075 835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e459b08a9904961afb97b31bc4778d0" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35269 } }
I20260811 17:14:54.500229 835 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:14:54.500295 835 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:14:54.500414 835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:54.501412 835 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e459b08a9904961afb97b31bc4778d0" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35269 } }
I20260811 17:14:54.501765 835 leader_election.cc:302] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e459b08a9904961afb97b31bc4778d0; no voters:
I20260811 17:14:54.502106 835 leader_election.cc:288] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:14:54.502307 841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:54.502599 841 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0 [term 1 LEADER]: Becoming Leader. State: Replica: 4e459b08a9904961afb97b31bc4778d0, State: Running, Role: LEADER
I20260811 17:14:54.502980 841 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e459b08a9904961afb97b31bc4778d0" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35269 } }
I20260811 17:14:54.503504 835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:54.504869 843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e459b08a9904961afb97b31bc4778d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e459b08a9904961afb97b31bc4778d0" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35269 } } }
I20260811 17:14:54.505017 843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:54.505380 844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e459b08a9904961afb97b31bc4778d0. Latest consensus state: current_term: 1 leader_uuid: "4e459b08a9904961afb97b31bc4778d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e459b08a9904961afb97b31bc4778d0" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 35269 } } }
I20260811 17:14:54.505484 852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:14:54.505492 844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0 [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:54.508412 852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:14:54.514432 852 catalog_manager.cc:1347] Generated new cluster ID: 79b4a96088144460a878a1362517f487
I20260811 17:14:54.514525 852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:14:54.536809 852 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:14:54.537189 852 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:14:54.547807 852 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e459b08a9904961afb97b31bc4778d0: Generated new TSK 0
I20260811 17:14:54.548686 852 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:14:54.553107 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:35269
--builtin_ntp_servers=127.21.54.148:46711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 17:14:54.687394 865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:54.687685 865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:54.687726 865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:54.691871 865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:54.691942 865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:54.692004 865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:14:54.696975 865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:46711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:35269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:54.698177 865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:54.699638 865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:54.709934 875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:54.709940 871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:54.709959 872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:54.710203 865 server_base.cc:1034] running on GCE node
I20260811 17:14:54.710727 865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:54.711447 865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:54.712639 865 hybrid_clock.cc:648] HybridClock initialized: now 1786468494712623 us; error 29 us; skew 500 ppm
I20260811 17:14:54.714581 865 webserver.cc:460] Webserver started at http://127.21.54.129:44445/ using document root <none> and password file <none>
I20260811 17:14:54.715191 865 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:54.715257 865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:54.715481 865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:54.717650 865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "ffb521f4346843c4825d5926c95e51bf"
format_stamp: "Formatted at 2026-08-11 17:14:54 on dist-test-slave-fd7l"
server_key: "c2173ed582600be96ac613327cde28fb"
server_key_iv: "eedad57ec23f854a860177fad2c25896"
server_key_version: "encryptionkey@0"
I20260811 17:14:54.718279 865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "ffb521f4346843c4825d5926c95e51bf"
format_stamp: "Formatted at 2026-08-11 17:14:54 on dist-test-slave-fd7l"
server_key: "c2173ed582600be96ac613327cde28fb"
server_key_iv: "eedad57ec23f854a860177fad2c25896"
server_key_version: "encryptionkey@0"
I20260811 17:14:54.722909 865 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:14:54.725802 881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:54.727216 865 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:14:54.727360 865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "ffb521f4346843c4825d5926c95e51bf"
format_stamp: "Formatted at 2026-08-11 17:14:54 on dist-test-slave-fd7l"
server_key: "c2173ed582600be96ac613327cde28fb"
server_key_iv: "eedad57ec23f854a860177fad2c25896"
server_key_version: "encryptionkey@0"
I20260811 17:14:54.727491 865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:54.732749 865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:54.733421 865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:54.733624 865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:54.734225 865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:54.735219 865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:54.735271 865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:54.735313 865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:54.735373 865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:54.745437 865 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:40417
I20260811 17:14:54.745501 1014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:40417 every 8 connection(s)
I20260811 17:14:54.746587 865 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:14:54.752235 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 865
I20260811 17:14:54.752377 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:14:54.752741 21722 external_mini_cluster.cc:1428] Setting key e83d14ffa84a21c340ec391856f402d1
I20260811 17:14:54.753645 839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:54.748795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:55605 (local address 127.21.54.190:35269)
0811 17:14:54.748985 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:54.748989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:54.750226 (+ 1237us) server_negotiation.cc:407] Connection header received
0811 17:14:54.751638 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:54.751642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:54.751688 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:54.751803 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:54.753141 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:54.753144 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:54.753147 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:54.753160 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:54.753183 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:54.753199 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:54.753304 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:54.753450 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 17:14:54.753496 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 17:14:54.754527 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:54.749273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35269 (local address 127.21.54.129:55605)
0811 17:14:54.750059 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:54.750101 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:54.751364 (+ 1263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:54.752000 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:54.752008 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:54.752105 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:54.752818 (+ 713us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:14:54.752820 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:54.752906 (+ 86us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:14:54.752981 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:54.753285 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:54.753296 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:54.753301 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 17:14:54.753498 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 17:14:54.753752 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":182,"threads_started":1}
I20260811 17:14:54.755232 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:35269
--builtin_ntp_servers=127.21.54.148:46711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 17:14:54.755995 1015 heartbeater.cc:347] Connected to a master server at 127.21.54.190:35269
I20260811 17:14:54.756253 1015 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:54.756776 1015 heartbeater.cc:511] Master 127.21.54.190:35269 requested a full tablet report, sending...
I20260811 17:14:54.758553 792 ts_manager.cc:194] Registered new tserver with Master: ffb521f4346843c4825d5926c95e51bf (127.21.54.129:40417)
W20260811 17:14:54.877405 1020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:54.877694 1020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:54.877735 1020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:54.881826 1020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:54.881922 1020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:54.882017 1020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:14:54.887128 1020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:46711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:35269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:54.888360 1020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:54.889853 1020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:54.900127 1026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:54.900177 1028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:54.900183 1025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:54.900624 1020 server_base.cc:1034] running on GCE node
I20260811 17:14:54.901115 1020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:54.901820 1020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:54.903023 1020 hybrid_clock.cc:648] HybridClock initialized: now 1786468494902996 us; error 36 us; skew 500 ppm
I20260811 17:14:54.905215 1020 webserver.cc:460] Webserver started at http://127.21.54.130:35881/ using document root <none> and password file <none>
I20260811 17:14:54.905889 1020 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:54.905985 1020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:54.906230 1020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:54.908427 1020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "916129e4df4748c5b737a6a47b9c2c1e"
format_stamp: "Formatted at 2026-08-11 17:14:54 on dist-test-slave-fd7l"
server_key: "163bcd8e81183cca62c10a784e4390d4"
server_key_iv: "eca378d48005640854bbbf2981aa53dc"
server_key_version: "encryptionkey@0"
I20260811 17:14:54.909053 1020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "916129e4df4748c5b737a6a47b9c2c1e"
format_stamp: "Formatted at 2026-08-11 17:14:54 on dist-test-slave-fd7l"
server_key: "163bcd8e81183cca62c10a784e4390d4"
server_key_iv: "eca378d48005640854bbbf2981aa53dc"
server_key_version: "encryptionkey@0"
I20260811 17:14:54.912894 1020 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:14:54.915691 1036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:54.917021 1020 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:54.917160 1020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "916129e4df4748c5b737a6a47b9c2c1e"
format_stamp: "Formatted at 2026-08-11 17:14:54 on dist-test-slave-fd7l"
server_key: "163bcd8e81183cca62c10a784e4390d4"
server_key_iv: "eca378d48005640854bbbf2981aa53dc"
server_key_version: "encryptionkey@0"
I20260811 17:14:54.917279 1020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:54.922356 1020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:54.922976 1020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:54.923175 1020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:54.923741 1020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:54.924746 1020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:54.924815 1020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:54.924886 1020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:54.924928 1020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:54.934279 1020 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:40893
I20260811 17:14:54.934300 1178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:40893 every 8 connection(s)
I20260811 17:14:54.935534 1020 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:14:54.942669 839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:54.937646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:46479 (local address 127.21.54.190:35269)
0811 17:14:54.937954 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:54.937958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:54.939051 (+ 1093us) server_negotiation.cc:407] Connection header received
0811 17:14:54.940774 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:54.940778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:54.940831 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:54.940941 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:54.942107 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:54.942111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:54.942114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:54.942127 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:54.942169 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:54.942178 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:54.942276 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:54.942432 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 17:14:54.942481 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210}
I20260811 17:14:54.943482 1182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:54.938071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35269 (local address 127.21.54.130:46479)
0811 17:14:54.938877 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:54.938921 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:54.940444 (+ 1523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:54.941124 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:54.941131 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:54.941235 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:54.941809 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:14:54.941811 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:54.941851 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:14:54.941926 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:54.942269 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:54.942279 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:54.942287 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 17:14:54.942475 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 17:14:54.942719 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":117,"threads_started":1}
I20260811 17:14:54.944759 1179 heartbeater.cc:347] Connected to a master server at 127.21.54.190:35269
I20260811 17:14:54.944811 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 1020
I20260811 17:14:54.944911 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:14:54.945048 1179 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:54.945245 21722 external_mini_cluster.cc:1428] Setting key 3c11e7a4ab3216e048eb20526469bafe
I20260811 17:14:54.945533 1179 heartbeater.cc:511] Master 127.21.54.190:35269 requested a full tablet report, sending...
I20260811 17:14:54.946659 793 ts_manager.cc:194] Registered new tserver with Master: 916129e4df4748c5b737a6a47b9c2c1e (127.21.54.130:40893)
I20260811 17:14:54.947256 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:35269
--builtin_ntp_servers=127.21.54.148:46711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 17:14:55.066124 1183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:55.066411 1183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:55.066452 1183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:55.070626 1183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:55.070690 1183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:55.070748 1183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:14:55.076157 1183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:46711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:35269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:55.077318 1183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:55.078717 1183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:55.088923 1191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:55.088973 1188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:55.089145 1183 server_base.cc:1034] running on GCE node
W20260811 17:14:55.088984 1189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:55.089659 1183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:55.090488 1183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:55.091678 1183 hybrid_clock.cc:648] HybridClock initialized: now 1786468495091662 us; error 31 us; skew 500 ppm
I20260811 17:14:55.093962 1183 webserver.cc:460] Webserver started at http://127.21.54.131:33749/ using document root <none> and password file <none>
I20260811 17:14:55.094655 1183 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:55.094726 1183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:55.095021 1183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:55.097568 1183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "450ac0cef8af4301903f0dca85015d32"
format_stamp: "Formatted at 2026-08-11 17:14:55 on dist-test-slave-fd7l"
server_key: "2794e6b4794f2184bb9ef65e3d1714b5"
server_key_iv: "a2cda5a4194c6fc4830903492d9e55a6"
server_key_version: "encryptionkey@0"
I20260811 17:14:55.098199 1183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "450ac0cef8af4301903f0dca85015d32"
format_stamp: "Formatted at 2026-08-11 17:14:55 on dist-test-slave-fd7l"
server_key: "2794e6b4794f2184bb9ef65e3d1714b5"
server_key_iv: "a2cda5a4194c6fc4830903492d9e55a6"
server_key_version: "encryptionkey@0"
I20260811 17:14:55.102355 1183 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:14:55.105515 1198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:55.106899 1183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:55.107030 1183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "450ac0cef8af4301903f0dca85015d32"
format_stamp: "Formatted at 2026-08-11 17:14:55 on dist-test-slave-fd7l"
server_key: "2794e6b4794f2184bb9ef65e3d1714b5"
server_key_iv: "a2cda5a4194c6fc4830903492d9e55a6"
server_key_version: "encryptionkey@0"
I20260811 17:14:55.107146 1183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:55.112113 1183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:55.112826 1183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:55.113009 1183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:55.113584 1183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:55.114593 1183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:55.114640 1183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:55.114701 1183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:55.114737 1183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:55.124086 1183 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:36941
I20260811 17:14:55.124167 1331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:36941 every 8 connection(s)
I20260811 17:14:55.125185 1183 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:14:55.131982 839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:55.127262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:53831 (local address 127.21.54.190:35269)
0811 17:14:55.127447 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:55.127452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:55.128618 (+ 1166us) server_negotiation.cc:407] Connection header received
0811 17:14:55.129986 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:55.129989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:55.130029 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:55.130109 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:55.131349 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:55.131352 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:55.131354 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:14:55.131367 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:55.131387 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:14:55.131394 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:55.131477 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:55.131752 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 17:14:55.131811 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 17:14:55.132966 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:55.127663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:35269 (local address 127.21.54.131:53831)
0811 17:14:55.128385 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:55.128427 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:55.129773 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:55.130281 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:55.130287 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:55.130379 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:55.131065 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:14:55.131067 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:55.131110 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:14:55.131185 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:55.131477 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:55.131484 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:55.131488 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 17:14:55.131725 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 17:14:55.131993 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":110,"threads_started":1}
I20260811 17:14:55.134232 1332 heartbeater.cc:347] Connected to a master server at 127.21.54.190:35269
I20260811 17:14:55.134495 1332 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:55.134883 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 1183
I20260811 17:14:55.134963 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:14:55.135032 1332 heartbeater.cc:511] Master 127.21.54.190:35269 requested a full tablet report, sending...
I20260811 17:14:55.135298 21722 external_mini_cluster.cc:1428] Setting key 0dbecc9e53650bae91b4dc74173d3e9f
I20260811 17:14:55.136287 792 ts_manager.cc:194] Registered new tserver with Master: 450ac0cef8af4301903f0dca85015d32 (127.21.54.131:36941)
I20260811 17:14:55.137439 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 17:14:55.140383 839 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:14:55.139376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60648 (local address 127.21.54.190:35269)
0811 17:14:55.139505 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:55.139509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:55.139744 (+ 235us) server_negotiation.cc:407] Connection header received
0811 17:14:55.139847 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:55.139849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:55.139879 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:55.139946 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:55.140188 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60648: BlockingRecv error: recv got EOF from 127.0.0.1:60648 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 17:14:55.140497 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 865
I20260811 17:14:55.153118 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 1020
I20260811 17:14:55.162056 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 1183
I20260811 17:14:55.173803 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 755
2026-08-11T17:14:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (902 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:14:55 dist-test-slave-fd7l krb5kdc[1345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:14:55 dist-test-slave-fd7l krb5kdc[1345](info): set up 2 sockets
Aug 11 17:14:55 dist-test-slave-fd7l krb5kdc[1345](info): commencing operation
krb5kdc: starting...
W20260811 17:14:56.339089 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:14:56 dist-test-slave-fd7l krb5kdc[1345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:14:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:14:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:14:56.480078 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:45051
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:45051
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:56.597695 1358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:56.598057 1358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:56.598125 1358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:56.602372 1358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:14:56.602481 1358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:56.602521 1358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:56.602586 1358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:14:56.602620 1358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:14:56.608069 1358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:45051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:45051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:56.609285 1358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:56.610369 1358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:56.619572 1363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:56.619576 1366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:56.619576 1364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:56.619967 1358 server_base.cc:1034] running on GCE node
I20260811 17:14:56.620702 1358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:56.621999 1358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:56.623224 1358 hybrid_clock.cc:648] HybridClock initialized: now 1786468496623189 us; error 50 us; skew 500 ppm
Aug 11 17:14:56 dist-test-slave-fd7l krb5kdc[1345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468496, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:56.626324 1358 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:14:56.627473 1358 webserver.cc:460] Webserver started at http://127.21.54.190:40951/ using document root <none> and password file <none>
I20260811 17:14:56.628105 1358 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:56.628206 1358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:56.628468 1358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:56.630728 1358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "86a12d08c1754b44bfc6f044a16abc0d"
format_stamp: "Formatted at 2026-08-11 17:14:56 on dist-test-slave-fd7l"
server_key: "aaf69e0dc61be741195773f28d42693e"
server_key_iv: "3edfbd747cf378a331d81506fa5e83c1"
server_key_version: "encryptionkey@0"
I20260811 17:14:56.631381 1358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "86a12d08c1754b44bfc6f044a16abc0d"
format_stamp: "Formatted at 2026-08-11 17:14:56 on dist-test-slave-fd7l"
server_key: "aaf69e0dc61be741195773f28d42693e"
server_key_iv: "3edfbd747cf378a331d81506fa5e83c1"
server_key_version: "encryptionkey@0"
I20260811 17:14:56.635600 1358 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 17:14:56.638486 1373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:56.639874 1358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:14:56.640054 1358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "86a12d08c1754b44bfc6f044a16abc0d"
format_stamp: "Formatted at 2026-08-11 17:14:56 on dist-test-slave-fd7l"
server_key: "aaf69e0dc61be741195773f28d42693e"
server_key_iv: "3edfbd747cf378a331d81506fa5e83c1"
server_key_version: "encryptionkey@0"
I20260811 17:14:56.640206 1358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:56.669507 1358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:56.672948 1358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:56.673193 1358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:56.681133 1358 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:45051
I20260811 17:14:56.681181 1433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:45051 every 8 connection(s)
I20260811 17:14:56.682230 1358 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:14:56.685322 1434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:56.688356 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 1358
I20260811 17:14:56.688457 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:14:56.688843 21722 external_mini_cluster.cc:1428] Setting key 80dcb427ec31cd6b337d59d8a7684314
I20260811 17:14:56.691645 1434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d: Bootstrap starting.
I20260811 17:14:56.694031 1434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:56.694794 1434 log.cc:824] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:56.696810 1434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d: No bootstrap required, opened a new log
Aug 11 17:14:56 dist-test-slave-fd7l krb5kdc[1345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:56.700763 1434 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a12d08c1754b44bfc6f044a16abc0d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45051 } }
I20260811 17:14:56.701027 1434 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:56.701108 1434 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86a12d08c1754b44bfc6f044a16abc0d, State: Initialized, Role: FOLLOWER
I20260811 17:14:56.701567 1434 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a12d08c1754b44bfc6f044a16abc0d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45051 } }
I20260811 17:14:56.701726 1434 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:14:56.701815 1434 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:14:56.701939 1434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:56.702886 1434 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a12d08c1754b44bfc6f044a16abc0d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45051 } }
I20260811 17:14:56.703264 1434 leader_election.cc:302] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86a12d08c1754b44bfc6f044a16abc0d; no voters:
I20260811 17:14:56.703604 1434 leader_election.cc:288] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:14:56.703734 1440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:56.703963 1440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d [term 1 LEADER]: Becoming Leader. State: Replica: 86a12d08c1754b44bfc6f044a16abc0d, State: Running, Role: LEADER
I20260811 17:14:56.704102 1438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:56.690271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44472 (local address 127.21.54.190:45051)
0811 17:14:56.690754 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:56.690770 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:56.690808 (+ 38us) server_negotiation.cc:407] Connection header received
0811 17:14:56.691472 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:56.691491 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:56.691874 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:56.692231 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:56.693166 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:56.694198 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:56.695148 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:56.695628 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:56.695797 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:56.698681 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:56.698687 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:56.698701 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:56.698729 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:56.700632 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:56.701146 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:56.701153 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:56.701160 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:56.701311 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:56.701645 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:56.701649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:56.701651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:56.702245 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:56.702477 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:56.702740 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 17:14:56.703074 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":161,"threads_started":1}
I20260811 17:14:56.704357 1440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a12d08c1754b44bfc6f044a16abc0d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45051 } }
I20260811 17:14:56.705054 1434 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:14:56.706221 1444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86a12d08c1754b44bfc6f044a16abc0d. Latest consensus state: current_term: 1 leader_uuid: "86a12d08c1754b44bfc6f044a16abc0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a12d08c1754b44bfc6f044a16abc0d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45051 } } }
I20260811 17:14:56.706429 1444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:56.706838 1451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:14:56.707391 1442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86a12d08c1754b44bfc6f044a16abc0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a12d08c1754b44bfc6f044a16abc0d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45051 } } }
I20260811 17:14:56.707489 1442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d [sys.catalog]: This master's current role is: LEADER
I20260811 17:14:56.710423 1451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:14:56.716202 1451 catalog_manager.cc:1347] Generated new cluster ID: e090d44aebe447729b4f1b57ee5a5dca
I20260811 17:14:56.716279 1451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:14:56.742205 1451 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:14:56.743619 1451 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:14:56.750029 1451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d: Generated new TSK 0
I20260811 17:14:56.750931 1451 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
I20260811 17:14:56.792246 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:45051
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:56.912135 1465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:56.912434 1465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:56.912480 1465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:56.916674 1465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:56.916749 1465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:56.916810 1465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:14:56.922318 1465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:45051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:56.923499 1465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:56.924638 1465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:56.935124 1473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:56.935142 1472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:56.935269 1465 server_base.cc:1034] running on GCE node
W20260811 17:14:56.935143 1475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:56.935968 1465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:56.936771 1465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:56.937963 1465 hybrid_clock.cc:648] HybridClock initialized: now 1786468496937948 us; error 41 us; skew 500 ppm
Aug 11 17:14:56 dist-test-slave-fd7l krb5kdc[1345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468496, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:56.941206 1465 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:14:56.942487 1465 webserver.cc:460] Webserver started at http://127.21.54.129:38093/ using document root <none> and password file <none>
I20260811 17:14:56.943287 1465 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:56.943369 1465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:56.943672 1465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:56.946308 1465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "f68b33365b624bf990c3959841ec2df4"
format_stamp: "Formatted at 2026-08-11 17:14:56 on dist-test-slave-fd7l"
server_key: "f14cbc925f09ce2654c811db80cdbe99"
server_key_iv: "3441667f8b81b06d9c94c125a75fcd74"
server_key_version: "encryptionkey@0"
I20260811 17:14:56.946916 1465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "f68b33365b624bf990c3959841ec2df4"
format_stamp: "Formatted at 2026-08-11 17:14:56 on dist-test-slave-fd7l"
server_key: "f14cbc925f09ce2654c811db80cdbe99"
server_key_iv: "3441667f8b81b06d9c94c125a75fcd74"
server_key_version: "encryptionkey@0"
I20260811 17:14:56.951062 1465 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:14:56.953994 1482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:56.955327 1465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:14:56.955528 1465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "f68b33365b624bf990c3959841ec2df4"
format_stamp: "Formatted at 2026-08-11 17:14:56 on dist-test-slave-fd7l"
server_key: "f14cbc925f09ce2654c811db80cdbe99"
server_key_iv: "3441667f8b81b06d9c94c125a75fcd74"
server_key_version: "encryptionkey@0"
I20260811 17:14:56.955641 1465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:56.971493 1465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:56.975190 1465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:56.975423 1465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:56.976096 1465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:56.977058 1465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:56.977108 1465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:56.977149 1465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:56.977165 1465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:56.986419 1465 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:38989
I20260811 17:14:56.986443 1610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:38989 every 8 connection(s)
I20260811 17:14:56.987377 1465 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:14:56.990813 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 1465
I20260811 17:14:56.990949 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:14:56.991345 21722 external_mini_cluster.cc:1428] Setting key db6696b87523e40c7ee23bf1aae794b3
Aug 11 17:14:56 dist-test-slave-fd7l krb5kdc[1345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468496, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.54.130@KRBTEST.COM; defaulting to no policy
I20260811 17:14:57.003698 1438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:56.989455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:51135 (local address 127.21.54.190:45051)
0811 17:14:56.989654 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:56.989657 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:56.990424 (+ 767us) server_negotiation.cc:407] Connection header received
0811 17:14:56.992349 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:56.992352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:56.992399 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:56.992479 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:56.994196 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:56.994772 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:56.996148 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:56.996584 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:56.996715 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:56.999624 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:56.999630 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:56.999636 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:56.999673 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:57.001489 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:57.002044 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:57.002049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:57.002052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:57.002170 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:57.002519 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:57.002529 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:57.002531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:57.002771 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:57.002882 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:57.003442 (+ 560us) server_negotiation.cc:299] Negotiation successful
0811 17:14:57.003565 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260811 17:14:57.004753 1614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:56.989806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45051 (local address 127.21.54.129:51135)
0811 17:14:56.990255 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:56.990299 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:56.992069 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:56.992719 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:56.992729 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:56.993206 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:56.993948 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:56.993972 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:56.994920 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:56.994927 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:56.995995 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:56.996005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:56.996737 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:56.996740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:56.996889 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:56.997580 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:56.997597 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:56.999417 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:57.001629 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:57.001635 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:57.001649 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:57.001929 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:57.002274 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:57.002276 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:57.002278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:57.002416 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:57.002878 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:57.002884 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:57.003201 (+ 317us) client_negotiation.cc:769] Sending connection context
0811 17:14:57.003375 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 17:14:57.003708 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":115,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.130@KRBTEST.COM" created.
I20260811 17:14:57.006326 1611 heartbeater.cc:347] Connected to a master server at 127.21.54.190:45051
I20260811 17:14:57.006623 1611 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:57.007233 1611 heartbeater.cc:511] Master 127.21.54.190:45051 requested a full tablet report, sending...
I20260811 17:14:57.008953 1393 ts_manager.cc:194] Registered new tserver with Master: f68b33365b624bf990c3959841ec2df4 (127.21.54.129:38989)
I20260811 17:14:57.010637 1393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:51135
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
I20260811 17:14:57.025462 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:45051
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:57.145951 1618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:57.146239 1618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:57.146286 1618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:57.150540 1618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:57.150611 1618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:57.150676 1618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:14:57.156167 1618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:45051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:57.157361 1618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:57.158385 1618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:57.168886 1624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:57.169065 1618 server_base.cc:1034] running on GCE node
W20260811 17:14:57.169134 1623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:57.168905 1626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:57.169759 1618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:57.170491 1618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:57.171674 1618 hybrid_clock.cc:648] HybridClock initialized: now 1786468497171647 us; error 30 us; skew 500 ppm
Aug 11 17:14:57 dist-test-slave-fd7l krb5kdc[1345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468497, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:57.180445 1618 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:14:57.181720 1618 webserver.cc:460] Webserver started at http://127.21.54.130:35311/ using document root <none> and password file <none>
I20260811 17:14:57.182412 1618 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:57.182485 1618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:57.182739 1618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:57.185304 1618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "defedc495fe648cca1fbd381a8ff5edd"
format_stamp: "Formatted at 2026-08-11 17:14:57 on dist-test-slave-fd7l"
server_key: "1fc5d5dfc22089cfc58fe2004ba9e98a"
server_key_iv: "3fbf2e6c244cbe9ba46963c88dc7a579"
server_key_version: "encryptionkey@0"
I20260811 17:14:57.185982 1618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "defedc495fe648cca1fbd381a8ff5edd"
format_stamp: "Formatted at 2026-08-11 17:14:57 on dist-test-slave-fd7l"
server_key: "1fc5d5dfc22089cfc58fe2004ba9e98a"
server_key_iv: "3fbf2e6c244cbe9ba46963c88dc7a579"
server_key_version: "encryptionkey@0"
I20260811 17:14:57.190235 1618 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:14:57.193121 1636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:57.194685 1618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:14:57.194839 1618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "defedc495fe648cca1fbd381a8ff5edd"
format_stamp: "Formatted at 2026-08-11 17:14:57 on dist-test-slave-fd7l"
server_key: "1fc5d5dfc22089cfc58fe2004ba9e98a"
server_key_iv: "3fbf2e6c244cbe9ba46963c88dc7a579"
server_key_version: "encryptionkey@0"
I20260811 17:14:57.194973 1618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:57.209735 1618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:57.213413 1618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:57.213697 1618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:57.214413 1618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:57.215363 1618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:57.215458 1618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:57.215575 1618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:57.215636 1618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:57.225574 1618 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:36579
I20260811 17:14:57.225639 1769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:36579 every 8 connection(s)
I20260811 17:14:57.226673 1618 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:14:57.234313 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 1618
I20260811 17:14:57.234441 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:14:57.234751 21722 external_mini_cluster.cc:1428] Setting key 35effff5e80aa3e5efa5c82a6183c3a0
Aug 11 17:14:57 dist-test-slave-fd7l krb5kdc[1345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468497, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:57.242525 1438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:57.228828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:49463 (local address 127.21.54.190:45051)
0811 17:14:57.229029 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:57.229033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:57.229918 (+ 885us) server_negotiation.cc:407] Connection header received
0811 17:14:57.231474 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:57.231477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:57.231542 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:57.231633 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:57.232945 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.233550 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.235364 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.235700 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.235813 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:57.238828 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:57.238831 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:57.238834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:57.238858 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:57.240374 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:57.240964 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:57.240967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:57.240968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:57.241051 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:57.241408 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:57.241411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:57.241413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:57.241602 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:57.241710 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:57.242232 (+ 522us) server_negotiation.cc:299] Negotiation successful
0811 17:14:57.242358 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 17:14:57.243341 1772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:57.229206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45051 (local address 127.21.54.130:49463)
0811 17:14:57.229749 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:57.229795 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:57.231237 (+ 1442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:57.231814 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:57.231822 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:57.232211 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:57.232780 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:57.232794 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.233867 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.233873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:57.235221 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:57.235232 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.235857 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.235859 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:57.236004 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:57.236803 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:57.236826 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:57.238642 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:57.240540 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:57.240546 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:57.240564 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:57.240842 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:57.241154 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:57.241157 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:57.241159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:57.241303 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:57.241709 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:57.241716 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:57.241971 (+ 255us) client_negotiation.cc:769] Sending connection context
0811 17:14:57.242175 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 17:14:57.242525 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":119,"threads_started":1}
I20260811 17:14:57.244707 1770 heartbeater.cc:347] Connected to a master server at 127.21.54.190:45051
I20260811 17:14:57.244989 1770 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:57.245728 1770 heartbeater.cc:511] Master 127.21.54.190:45051 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.54.131@KRBTEST.COM; defaulting to no policy
I20260811 17:14:57.246982 1393 ts_manager.cc:194] Registered new tserver with Master: defedc495fe648cca1fbd381a8ff5edd (127.21.54.130:36579)
I20260811 17:14:57.247653 1393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:49463
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
I20260811 17:14:57.268206 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:45051
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:14:57.395291 1775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:14:57.395579 1775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:14:57.395624 1775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:14:57.399868 1775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:14:57.399942 1775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:14:57.400003 1775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:14:57.405323 1775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:45051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:14:57.406551 1775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:14:57.407614 1775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:14:57.417269 1783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:57.417272 1781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:14:57.417332 1785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:14:57.417706 1775 server_base.cc:1034] running on GCE node
I20260811 17:14:57.418118 1775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:14:57.418797 1775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:14:57.420009 1775 hybrid_clock.cc:648] HybridClock initialized: now 1786468497419953 us; error 70 us; skew 500 ppm
Aug 11 17:14:57 dist-test-slave-fd7l krb5kdc[1345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468497, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:14:57.423179 1775 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:14:57.424157 1775 webserver.cc:460] Webserver started at http://127.21.54.131:38843/ using document root <none> and password file <none>
I20260811 17:14:57.424850 1775 fs_manager.cc:359] Metadata directory not provided
I20260811 17:14:57.424948 1775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:14:57.425204 1775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:14:57.427330 1775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "426ea908e98d4e5ab9b7b82d5f26e80c"
format_stamp: "Formatted at 2026-08-11 17:14:57 on dist-test-slave-fd7l"
server_key: "a15ec326d079f61f034e5273ba8e05bf"
server_key_iv: "5547e7e56a0d048fb641a89dedaa6f5d"
server_key_version: "encryptionkey@0"
I20260811 17:14:57.427914 1775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "426ea908e98d4e5ab9b7b82d5f26e80c"
format_stamp: "Formatted at 2026-08-11 17:14:57 on dist-test-slave-fd7l"
server_key: "a15ec326d079f61f034e5273ba8e05bf"
server_key_iv: "5547e7e56a0d048fb641a89dedaa6f5d"
server_key_version: "encryptionkey@0"
I20260811 17:14:57.431869 1775 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:14:57.434541 1792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:57.435883 1775 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 17:14:57.436074 1775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "426ea908e98d4e5ab9b7b82d5f26e80c"
format_stamp: "Formatted at 2026-08-11 17:14:57 on dist-test-slave-fd7l"
server_key: "a15ec326d079f61f034e5273ba8e05bf"
server_key_iv: "5547e7e56a0d048fb641a89dedaa6f5d"
server_key_version: "encryptionkey@0"
I20260811 17:14:57.436200 1775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:14:57.457653 1775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:14:57.461354 1775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:14:57.461589 1775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:14:57.462261 1775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:14:57.463610 1775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:14:57.463690 1775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:57.463763 1775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:14:57.463795 1775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:14:57.475502 1775 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:37159
I20260811 17:14:57.475519 1922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:37159 every 8 connection(s)
I20260811 17:14:57.476464 1775 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:14:57.477528 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 1775
I20260811 17:14:57.477689 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:14:57.478039 21722 external_mini_cluster.cc:1428] Setting key 8b74e90cfa53dc352964785990a42f95
Aug 11 17:14:57 dist-test-slave-fd7l krb5kdc[1345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468497, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:14:57.492309 1438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:57.478624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:37553 (local address 127.21.54.190:45051)
0811 17:14:57.478778 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:57.478782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:57.479821 (+ 1039us) server_negotiation.cc:407] Connection header received
0811 17:14:57.481778 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:57.481782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:57.481842 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:57.481930 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:57.483241 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.483840 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.485191 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.485445 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.485569 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:57.488341 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:57.488345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:57.488347 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:57.488376 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:57.489945 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:57.490465 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:57.490468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:57.490469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:57.490644 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:57.491050 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:57.491055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:57.491057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:57.491330 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:57.491520 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:57.492009 (+ 489us) server_negotiation.cc:299] Negotiation successful
0811 17:14:57.492140 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 17:14:57.493343 1926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:57.479130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45051 (local address 127.21.54.131:37553)
0811 17:14:57.479679 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:57.479732 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:57.481530 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:57.482118 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:57.482126 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:57.482553 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:14:57.483080 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:57.483095 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.483982 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.483987 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:57.485053 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:57.485064 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.485619 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.485623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:57.485814 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:57.486418 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:14:57.486435 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:14:57.488174 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:14:57.490066 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:57.490071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:57.490085 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:57.490369 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:57.490749 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:14:57.490752 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:14:57.490753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:14:57.490911 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:14:57.491471 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:14:57.491478 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:14:57.491759 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 17:14:57.491974 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 17:14:57.492332 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":139,"threads_started":1}
I20260811 17:14:57.494472 1924 heartbeater.cc:347] Connected to a master server at 127.21.54.190:45051
I20260811 17:14:57.494752 1924 heartbeater.cc:464] Registering TS with master...
I20260811 17:14:57.495265 1924 heartbeater.cc:511] Master 127.21.54.190:45051 requested a full tablet report, sending...
I20260811 17:14:57.496344 1393 ts_manager.cc:194] Registered new tserver with Master: 426ea908e98d4e5ab9b7b82d5f26e80c (127.21.54.131:37159)
I20260811 17:14:57.496979 1393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:37553
I20260811 17:14:57.504757 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:14:57.515129 1438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:57.507007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44478 (local address 127.21.54.190:45051)
0811 17:14:57.507158 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:57.507161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:57.507313 (+ 152us) server_negotiation.cc:407] Connection header received
0811 17:14:57.507433 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:57.507435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:57.507478 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:57.507551 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:14:57.508376 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.509003 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.509855 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.510061 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.510214 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:57.511736 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:14:57.511739 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:14:57.511741 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:14:57.511768 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:14:57.513391 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:57.513844 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:57.513847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:57.513848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:57.513935 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:14:57.514268 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:14:57.514271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:14:57.514272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:14:57.514568 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:14:57.514686 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:57.514894 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 17:14:57.515005 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 17:14:57.518592 1393 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:14:57.522037 1393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:14:57.538133 1939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:57.531770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:37159 (local address 127.0.0.1:41476)
0811 17:14:57.532678 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:57.532739 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:57.532890 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:57.533860 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:57.533863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:57.533882 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:57.533993 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:57.534002 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.535478 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.535491 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:57.536700 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:57.536708 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.537543 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.537547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:57.537665 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:57.537704 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 17:14:57.537818 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 17:14:57.537951 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":70,"spinlock_wait_cycles":5760,"thread_start_us":98,"threads_started":1}
I20260811 17:14:57.538308 1940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:57.532584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57312 (local address 127.21.54.130:36579)
0811 17:14:57.533178 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:57.533189 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:57.533212 (+ 23us) server_negotiation.cc:407] Connection header received
0811 17:14:57.533298 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:57.533305 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:57.533528 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:57.534079 (+ 551us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:57.534251 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.534961 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.536631 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.537508 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.537620 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:57.537778 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:57.538030 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 17:14:57.538152 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":157,"threads_started":1}
I20260811 17:14:57.538697 1941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:57.532652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58062 (local address 127.21.54.129:38989)
0811 17:14:57.533063 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:57.533072 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:57.533318 (+ 246us) server_negotiation.cc:407] Connection header received
0811 17:14:57.533510 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:57.533515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:57.533697 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:57.533809 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:57.534216 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.535354 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.536827 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.537625 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.537717 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:57.537844 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:57.538399 (+ 555us) server_negotiation.cc:299] Negotiation successful
0811 17:14:57.538584 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":105,"threads_started":1}
I20260811 17:14:57.538908 1938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:57.531964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41476 (local address 127.21.54.131:37159)
0811 17:14:57.533364 (+ 1400us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:57.533377 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:57.533399 (+ 22us) server_negotiation.cc:407] Connection header received
0811 17:14:57.533484 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:57.533490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:57.533697 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:57.533884 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:57.534100 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.535323 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.536817 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.537408 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.537543 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:57.537706 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:57.538212 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 17:14:57.538720 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1293,"thread_start_us":176,"threads_started":1}
I20260811 17:14:57.539007 1937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:57.532286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:38989 (local address 127.0.0.1:58062)
0811 17:14:57.533228 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:57.533243 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:57.533381 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:57.533967 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:57.533972 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:57.533993 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:57.534093 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:57.534099 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.535516 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.535519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:57.536686 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:57.536698 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.537909 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.537912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:57.538037 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:57.538062 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 17:14:57.538133 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 17:14:57.538209 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":881,"mutex_wait_us":4,"thread_start_us":103,"threads_started":1}
I20260811 17:14:57.538134 1936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:57.531987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:36579 (local address 127.0.0.1:57312)
0811 17:14:57.532677 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:57.532737 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:57.532890 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:57.533725 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:57.533733 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:57.533763 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:57.533929 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:57.533938 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.535112 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.535117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:57.536487 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:57.536499 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.537790 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.537796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:57.537880 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:57.537899 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 17:14:57.537970 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 17:14:57.538022 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":1,"spinlock_wait_cycles":6656,"thread_start_us":131,"threads_started":1}
I20260811 17:14:57.541960 1846 tablet_service.cc:1467] Processing CreateTablet for tablet 4415aaf88e9e41529f7ba60321a5786a (DEFAULT_TABLE table=test-table [id=31abcc1bec864c02b66797991b17c1dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:57.542105 1537 tablet_service.cc:1467] Processing CreateTablet for tablet 4415aaf88e9e41529f7ba60321a5786a (DEFAULT_TABLE table=test-table [id=31abcc1bec864c02b66797991b17c1dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:57.542109 1694 tablet_service.cc:1467] Processing CreateTablet for tablet 4415aaf88e9e41529f7ba60321a5786a (DEFAULT_TABLE table=test-table [id=31abcc1bec864c02b66797991b17c1dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:14:57.543109 1846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4415aaf88e9e41529f7ba60321a5786a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:57.543176 1694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4415aaf88e9e41529f7ba60321a5786a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:57.543185 1537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4415aaf88e9e41529f7ba60321a5786a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:14:57.550529 1943 tablet_bootstrap.cc:492] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c: Bootstrap starting.
I20260811 17:14:57.551234 1944 tablet_bootstrap.cc:492] T 4415aaf88e9e41529f7ba60321a5786a P f68b33365b624bf990c3959841ec2df4: Bootstrap starting.
I20260811 17:14:57.551474 1945 tablet_bootstrap.cc:492] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd: Bootstrap starting.
I20260811 17:14:57.553331 1944 tablet_bootstrap.cc:654] T 4415aaf88e9e41529f7ba60321a5786a P f68b33365b624bf990c3959841ec2df4: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:57.553386 1943 tablet_bootstrap.cc:654] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:57.554019 1945 tablet_bootstrap.cc:654] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd: Neither blocks nor log segments found. Creating new log.
I20260811 17:14:57.554167 1944 log.cc:824] T 4415aaf88e9e41529f7ba60321a5786a P f68b33365b624bf990c3959841ec2df4: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:57.554662 1943 log.cc:824] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:57.554764 1945 log.cc:824] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd: Log is configured to *not* fsync() on all Append() calls
I20260811 17:14:57.556589 1944 tablet_bootstrap.cc:492] T 4415aaf88e9e41529f7ba60321a5786a P f68b33365b624bf990c3959841ec2df4: No bootstrap required, opened a new log
I20260811 17:14:57.556799 1943 tablet_bootstrap.cc:492] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c: No bootstrap required, opened a new log
I20260811 17:14:57.556813 1944 ts_tablet_manager.cc:1397] T 4415aaf88e9e41529f7ba60321a5786a P f68b33365b624bf990c3959841ec2df4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 17:14:57.556946 1945 tablet_bootstrap.cc:492] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd: No bootstrap required, opened a new log
I20260811 17:14:57.557024 1943 ts_tablet_manager.cc:1397] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 17:14:57.557140 1945 ts_tablet_manager.cc:1397] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 17:14:57.560158 1943 raft_consensus.cc:348] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "426ea908e98d4e5ab9b7b82d5f26e80c" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37159 } } peers { permanent_uuid: "defedc495fe648cca1fbd381a8ff5edd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36579 } } peers { permanent_uuid: "f68b33365b624bf990c3959841ec2df4" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38989 } }
I20260811 17:14:57.560477 1943 raft_consensus.cc:374] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:57.560562 1943 raft_consensus.cc:729] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 426ea908e98d4e5ab9b7b82d5f26e80c, State: Initialized, Role: FOLLOWER
I20260811 17:14:57.561049 1943 consensus_queue.cc:260] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "426ea908e98d4e5ab9b7b82d5f26e80c" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37159 } } peers { permanent_uuid: "defedc495fe648cca1fbd381a8ff5edd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36579 } } peers { permanent_uuid: "f68b33365b624bf990c3959841ec2df4" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38989 } }
I20260811 17:14:57.561457 1945 raft_consensus.cc:348] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "426ea908e98d4e5ab9b7b82d5f26e80c" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37159 } } peers { permanent_uuid: "defedc495fe648cca1fbd381a8ff5edd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36579 } } peers { permanent_uuid: "f68b33365b624bf990c3959841ec2df4" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38989 } }
I20260811 17:14:57.561571 1944 raft_consensus.cc:348] T 4415aaf88e9e41529f7ba60321a5786a P f68b33365b624bf990c3959841ec2df4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "426ea908e98d4e5ab9b7b82d5f26e80c" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37159 } } peers { permanent_uuid: "defedc495fe648cca1fbd381a8ff5edd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36579 } } peers { permanent_uuid: "f68b33365b624bf990c3959841ec2df4" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38989 } }
I20260811 17:14:57.561726 1945 raft_consensus.cc:374] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:57.561761 1924 heartbeater.cc:503] Master 127.21.54.190:45051 was elected leader, sending a full tablet report...
I20260811 17:14:57.561784 1944 raft_consensus.cc:374] T 4415aaf88e9e41529f7ba60321a5786a P f68b33365b624bf990c3959841ec2df4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:14:57.561777 1945 raft_consensus.cc:729] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: defedc495fe648cca1fbd381a8ff5edd, State: Initialized, Role: FOLLOWER
I20260811 17:14:57.561841 1944 raft_consensus.cc:729] T 4415aaf88e9e41529f7ba60321a5786a P f68b33365b624bf990c3959841ec2df4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f68b33365b624bf990c3959841ec2df4, State: Initialized, Role: FOLLOWER
I20260811 17:14:57.562182 1943 ts_tablet_manager.cc:1428] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260811 17:14:57.562242 1945 consensus_queue.cc:260] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "426ea908e98d4e5ab9b7b82d5f26e80c" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37159 } } peers { permanent_uuid: "defedc495fe648cca1fbd381a8ff5edd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36579 } } peers { permanent_uuid: "f68b33365b624bf990c3959841ec2df4" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38989 } }
I20260811 17:14:57.562237 1944 consensus_queue.cc:260] T 4415aaf88e9e41529f7ba60321a5786a P f68b33365b624bf990c3959841ec2df4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "426ea908e98d4e5ab9b7b82d5f26e80c" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37159 } } peers { permanent_uuid: "defedc495fe648cca1fbd381a8ff5edd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36579 } } peers { permanent_uuid: "f68b33365b624bf990c3959841ec2df4" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38989 } }
I20260811 17:14:57.563205 1944 ts_tablet_manager.cc:1428] T 4415aaf88e9e41529f7ba60321a5786a P f68b33365b624bf990c3959841ec2df4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 17:14:57.563205 1945 ts_tablet_manager.cc:1428] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 17:14:57.563558 1611 heartbeater.cc:503] Master 127.21.54.190:45051 was elected leader, sending a full tablet report...
I20260811 17:14:57.564296 1770 heartbeater.cc:503] Master 127.21.54.190:45051 was elected leader, sending a full tablet report...
I20260811 17:14:57.602319 1950 raft_consensus.cc:482] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:14:57.602553 1950 raft_consensus.cc:504] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "426ea908e98d4e5ab9b7b82d5f26e80c" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37159 } } peers { permanent_uuid: "defedc495fe648cca1fbd381a8ff5edd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36579 } } peers { permanent_uuid: "f68b33365b624bf990c3959841ec2df4" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38989 } }
I20260811 17:14:57.603775 1950 leader_election.cc:288] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 426ea908e98d4e5ab9b7b82d5f26e80c (127.21.54.131:37159), f68b33365b624bf990c3959841ec2df4 (127.21.54.129:38989)
I20260811 17:14:57.608198 1938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:57.604126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:48573 (local address 127.21.54.131:37159)
0811 17:14:57.604268 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:57.604273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:57.604387 (+ 114us) server_negotiation.cc:407] Connection header received
0811 17:14:57.604572 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:57.604575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:57.604619 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:57.604705 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:57.605023 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.605620 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.606934 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.607587 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.607711 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:57.607767 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:57.607966 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 17:14:57.608059 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 17:14:57.608175 1772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:57.604028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:37159 (local address 127.21.54.130:48573)
0811 17:14:57.604250 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:57.604266 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:57.604395 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:57.604728 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:57.604731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:57.604749 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:57.604907 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:57.604915 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.605725 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.605730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:57.606803 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:57.606813 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.607714 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.607717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:57.607799 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:57.607849 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 17:14:57.607928 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 17:14:57.607998 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
I20260811 17:14:57.608305 1941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:57.604261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:60243 (local address 127.21.54.129:38989)
0811 17:14:57.604392 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:57.604397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:57.604668 (+ 271us) server_negotiation.cc:407] Connection header received
0811 17:14:57.604825 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:57.604828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:57.604909 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:57.604996 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:14:57.605265 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.605879 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.607116 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.607757 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.607872 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:57.607942 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:57.608098 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 17:14:57.608178 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 17:14:57.608319 1952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:57.604055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:38989 (local address 127.21.54.130:60243)
0811 17:14:57.604565 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:57.604584 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:14:57.604709 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:14:57.605006 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:14:57.605009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:14:57.605026 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:14:57.605142 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:57.605150 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.606038 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.606040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:57.606970 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:14:57.606980 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.607874 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.607880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:14:57.607979 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:57.607999 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 17:14:57.608071 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 17:14:57.608129 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":131,"threads_started":1}
I20260811 17:14:57.609094 1558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4415aaf88e9e41529f7ba60321a5786a" candidate_uuid: "defedc495fe648cca1fbd381a8ff5edd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f68b33365b624bf990c3959841ec2df4" is_pre_election: true
I20260811 17:14:57.609138 1867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4415aaf88e9e41529f7ba60321a5786a" candidate_uuid: "defedc495fe648cca1fbd381a8ff5edd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "426ea908e98d4e5ab9b7b82d5f26e80c" is_pre_election: true
I20260811 17:14:57.609438 1558 raft_consensus.cc:2454] T 4415aaf88e9e41529f7ba60321a5786a P f68b33365b624bf990c3959841ec2df4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate defedc495fe648cca1fbd381a8ff5edd in term 0.
I20260811 17:14:57.609448 1867 raft_consensus.cc:2454] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate defedc495fe648cca1fbd381a8ff5edd in term 0.
I20260811 17:14:57.609952 1637 leader_election.cc:302] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: defedc495fe648cca1fbd381a8ff5edd, f68b33365b624bf990c3959841ec2df4; no voters:
I20260811 17:14:57.610242 1950 raft_consensus.cc:2781] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:14:57.610332 1950 raft_consensus.cc:482] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:14:57.610376 1950 raft_consensus.cc:3037] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:57.611449 1950 raft_consensus.cc:504] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "426ea908e98d4e5ab9b7b82d5f26e80c" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37159 } } peers { permanent_uuid: "defedc495fe648cca1fbd381a8ff5edd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36579 } } peers { permanent_uuid: "f68b33365b624bf990c3959841ec2df4" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38989 } }
I20260811 17:14:57.611837 1950 leader_election.cc:288] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [CANDIDATE]: Term 1 election: Requested vote from peers 426ea908e98d4e5ab9b7b82d5f26e80c (127.21.54.131:37159), f68b33365b624bf990c3959841ec2df4 (127.21.54.129:38989)
I20260811 17:14:57.612144 1867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4415aaf88e9e41529f7ba60321a5786a" candidate_uuid: "defedc495fe648cca1fbd381a8ff5edd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "426ea908e98d4e5ab9b7b82d5f26e80c"
I20260811 17:14:57.612190 1558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4415aaf88e9e41529f7ba60321a5786a" candidate_uuid: "defedc495fe648cca1fbd381a8ff5edd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f68b33365b624bf990c3959841ec2df4"
I20260811 17:14:57.612313 1558 raft_consensus.cc:3037] T 4415aaf88e9e41529f7ba60321a5786a P f68b33365b624bf990c3959841ec2df4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:57.612319 1867 raft_consensus.cc:3037] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:14:57.613438 1558 raft_consensus.cc:2454] T 4415aaf88e9e41529f7ba60321a5786a P f68b33365b624bf990c3959841ec2df4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate defedc495fe648cca1fbd381a8ff5edd in term 1.
I20260811 17:14:57.613538 1867 raft_consensus.cc:2454] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate defedc495fe648cca1fbd381a8ff5edd in term 1.
I20260811 17:14:57.613847 1640 leader_election.cc:302] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 426ea908e98d4e5ab9b7b82d5f26e80c, defedc495fe648cca1fbd381a8ff5edd; no voters:
I20260811 17:14:57.614042 1950 raft_consensus.cc:2781] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:14:57.614300 1950 raft_consensus.cc:686] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [term 1 LEADER]: Becoming Leader. State: Replica: defedc495fe648cca1fbd381a8ff5edd, State: Running, Role: LEADER
I20260811 17:14:57.614643 1950 consensus_queue.cc:237] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "426ea908e98d4e5ab9b7b82d5f26e80c" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37159 } } peers { permanent_uuid: "defedc495fe648cca1fbd381a8ff5edd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36579 } } peers { permanent_uuid: "f68b33365b624bf990c3959841ec2df4" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38989 } }
I20260811 17:14:57.618083 1392 catalog_manager.cc:5495] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd reported cstate change: term changed from 0 to 1, leader changed from <none> to defedc495fe648cca1fbd381a8ff5edd (127.21.54.130). New cstate: current_term: 1 leader_uuid: "defedc495fe648cca1fbd381a8ff5edd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "426ea908e98d4e5ab9b7b82d5f26e80c" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "defedc495fe648cca1fbd381a8ff5edd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 36579 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f68b33365b624bf990c3959841ec2df4" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38989 } health_report { overall_health: UNKNOWN } } }
I20260811 17:14:57.641460 1940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:57.637210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57314 (local address 127.21.54.130:36579)
0811 17:14:57.637380 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 17:14:57.637386 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:14:57.637402 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:14:57.637592 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:14:57.637602 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:14:57.637671 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:14:57.637809 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:14:57.638108 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.638666 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.639692 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:14:57.639943 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:14:57.640138 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:14:57.640525 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:14:57.640998 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:14:57.641161 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 17:14:57.641208 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 17:14:57.641272 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 17:14:57.648229 1558 raft_consensus.cc:1261] T 4415aaf88e9e41529f7ba60321a5786a P f68b33365b624bf990c3959841ec2df4 [term 1 FOLLOWER]: Refusing update from remote peer defedc495fe648cca1fbd381a8ff5edd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:57.648229 1867 raft_consensus.cc:1261] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c [term 1 FOLLOWER]: Refusing update from remote peer defedc495fe648cca1fbd381a8ff5edd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:14:57.649161 1950 consensus_queue.cc:1035] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [LEADER]: Connected to new peer: Peer: permanent_uuid: "426ea908e98d4e5ab9b7b82d5f26e80c" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:57.649398 1953 consensus_queue.cc:1035] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f68b33365b624bf990c3959841ec2df4" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:14:57.659137 1957 mvcc.cc:205] Tried to move back new op lower bound from 7317374966358708224 to 7317374966232338432. Current Snapshot: MvccSnapshot[applied={T|T < 7317374966358708224}]
I20260811 17:14:57.667373 1392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:57.667591 1392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:14:57.670414 1392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86a12d08c1754b44bfc6f044a16abc0d: Sending DeleteTablet for 3 replicas of tablet 4415aaf88e9e41529f7ba60321a5786a
I20260811 17:14:57.671306 1694 tablet_service.cc:1514] Processing DeleteTablet for tablet 4415aaf88e9e41529f7ba60321a5786a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:57 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:57312
I20260811 17:14:57.671355 1846 tablet_service.cc:1514] Processing DeleteTablet for tablet 4415aaf88e9e41529f7ba60321a5786a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:57 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:41476
I20260811 17:14:57.671588 1537 tablet_service.cc:1514] Processing DeleteTablet for tablet 4415aaf88e9e41529f7ba60321a5786a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:14:57 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:58062
I20260811 17:14:57.672302 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 1465
I20260811 17:14:57.672355 1968 tablet_replica.cc:330] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd: stopping tablet replica
I20260811 17:14:57.675993 1968 raft_consensus.cc:2229] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:14:57.676772 1968 raft_consensus.cc:2258] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:57.678761 1968 ts_tablet_manager.cc:1905] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:57.680805 1969 tablet_replica.cc:330] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c: stopping tablet replica
I20260811 17:14:57.681210 1969 raft_consensus.cc:2229] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:14:57.681579 1969 raft_consensus.cc:2258] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:14:57.681619 1968 ts_tablet_manager.cc:1918] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:14:57.681722 1968 log.cc:1196] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/4415aaf88e9e41529f7ba60321a5786a
I20260811 17:14:57.682067 1968 ts_tablet_manager.cc:1939] T 4415aaf88e9e41529f7ba60321a5786a P defedc495fe648cca1fbd381a8ff5edd: Deleting consensus metadata
I20260811 17:14:57.683439 1969 ts_tablet_manager.cc:1905] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:14:57.683439 1378 catalog_manager.cc:4841] TS defedc495fe648cca1fbd381a8ff5edd (127.21.54.130:36579): tablet 4415aaf88e9e41529f7ba60321a5786a (table test-table [id=31abcc1bec864c02b66797991b17c1dc]) successfully deleted
I20260811 17:14:57.686993 1969 ts_tablet_manager.cc:1918] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:14:57.687099 1969 log.cc:1196] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/4415aaf88e9e41529f7ba60321a5786a
I20260811 17:14:57.687453 1969 ts_tablet_manager.cc:1939] T 4415aaf88e9e41529f7ba60321a5786a P 426ea908e98d4e5ab9b7b82d5f26e80c: Deleting consensus metadata
W20260811 17:14:57.689278 1375 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.54.129:38989 (error 108)
I20260811 17:14:57.689569 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 1618
I20260811 17:14:57.689706 1379 catalog_manager.cc:4841] TS 426ea908e98d4e5ab9b7b82d5f26e80c (127.21.54.131:37159): tablet 4415aaf88e9e41529f7ba60321a5786a (table test-table [id=31abcc1bec864c02b66797991b17c1dc]) successfully deleted
I20260811 17:14:57.693555 1936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:14:57.693205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:38989 (local address 127.0.0.1:58072)
0811 17:14:57.693385 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 17:14:57.693471 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.129:38989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260811 17:14:57.693871 1375 catalog_manager.cc:4584] TS f68b33365b624bf990c3959841ec2df4 (127.21.54.129:38989): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4415aaf88e9e41529f7ba60321a5786a: Network error: Client connection negotiation failed: client connection to 127.21.54.129:38989: connect: Connection refused (error 111)
I20260811 17:14:57.701009 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 1775
I20260811 17:14:57.711573 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 1358
2026-08-11T17:14:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2537 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:14:57 dist-test-slave-fd7l krb5kdc[1974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:14:57 dist-test-slave-fd7l krb5kdc[1974](info): set up 2 sockets
Aug 11 17:14:57 dist-test-slave-fd7l krb5kdc[1974](info): commencing operation
krb5kdc: starting...
W20260811 17:14:58.883208 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:14:58 dist-test-slave-fd7l krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:14:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:14:58Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.54.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.54.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.54.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/rangeradmin_127.21.54.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.54.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/rangeradmin_127.21.54.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.54.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.54.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.54.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/rangerlookup_127.21.54.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.54.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/rangerlookup_127.21.54.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.54.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.54.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.54.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/HTTP_127.21.54.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.54.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/HTTP_127.21.54.148@KRBTEST.COM.keytab.
I20260811 17:14:59.081386 21722 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task2dz75L/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/postgres -l logfile start
2026-08-11 17:15:01.901 UTC [2012] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 17:15:01.901 UTC [2012] LOG: listening on IPv4 address "127.21.54.148", port 33987
2026-08-11 17:15:01.905 UTC [2012] LOG: listening on Unix socket "/tmp/.s.PGSQL.33987"
2026-08-11 17:15:01.929 UTC [2015] LOG: database system was shut down at 2026-08-11 17:15:00 UTC
2026-08-11 17:15:01.933 UTC [2012] LOG: database system is ready to accept connections
I20260811 17:15:02.991647 21722 mini_postgres.cc:96] Postgres bound to 33987
2026-08-11 17:15:03.000 UTC [2024] FATAL: database "slave" does not exist
127.21.54.148:33987 - accepting connections
I20260811 17:15:03.000896 21722 mini_ranger.cc:161] Starting Ranger...
I20260811 17:15:03.018795 21722 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 17:15:03.590384 21722 mini_ranger.cc:87] Created ranger Postgres database
I20260811 17:15:03.590487 21722 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ranger-admin
2026-08-11 17:15:03,890 [I] DB FLAVOR :POSTGRES
2026-08-11 17:15:03,891 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 17:15:03,891 [I] Checking connection..
2026-08-11 17:15:03,891 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 17:15:04,176 [I] Checking connection passed.
2026-08-11 17:15:04,176 [I] --------- Verifying version history table ---------
2026-08-11 17:15:04,176 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 17:15:04,456 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 17:15:04,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 17:15:04,753 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 17:15:04,753 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 17:15:04,753 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 17:15:05.036 UTC [2096] WARNING: there is no transaction in progress
2026-08-11 17:15:05,043 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 17:15:05,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 17:15:05,307 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 17:15:05,308 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 17:15:05,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 17:15:05,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 17:15:05,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-fd7l.c.gcp-upstream.internal','N') ;"
2026-08-11 17:15:06,088 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 17:15:06,366 [I] Table x_portal_user does not exist in database ranger
2026-08-11 17:15:06,367 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 17:15:06,632 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 17:15:06,633 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 17:15:06,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 17:15:07,158 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 17:15:07,158 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 17:15:08.299 UTC [2217] WARNING: there is no transaction in progress
2026-08-11 17:15:08.318 UTC [2217] WARNING: there is no transaction in progress
2026-08-11 17:15:08.340 UTC [2217] WARNING: there is no transaction in progress
2026-08-11 17:15:08.359 UTC [2217] WARNING: there is no transaction in progress
2026-08-11 17:15:08.381 UTC [2217] WARNING: there is no transaction in progress
2026-08-11 17:15:08.406 UTC [2217] WARNING: there is no transaction in progress
2026-08-11 17:15:08.479 UTC [2217] WARNING: there is no transaction in progress
2026-08-11 17:15:08.488 UTC [2217] WARNING: there is no transaction in progress
2026-08-11 17:15:08.505 UTC [2217] WARNING: there is no transaction in progress
2026-08-11 17:15:08.518 UTC [2217] WARNING: there is no transaction in progress
2026-08-11 17:15:08.531 UTC [2217] WARNING: there is no transaction in progress
2026-08-11 17:15:08.548 UTC [2217] WARNING: there is no transaction in progress
2026-08-11 17:15:08.560 UTC [2217] WARNING: there is no transaction in progress
2026-08-11 17:15:08.569 UTC [2217] WARNING: there is no transaction in progress
2026-08-11 17:15:08.578 UTC [2217] WARNING: there is no transaction in progress
2026-08-11 17:15:08.587 UTC [2217] WARNING: there is no transaction in progress
2026-08-11 17:15:09,426 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 17:15:09,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 17:15:09,716 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 17:15:09,716 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 17:15:10,001 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 17:15:10,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 17:15:10,285 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 17:15:10,285 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 17:15:10,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 17:15:10,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-fd7l.c.gcp-upstream.internal';"
2026-08-11 17:15:11,068 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 17:15:11,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 17:15:11,323 [I] DB_PATCHES have already been applied
I20260811 17:15:11.329627 21722 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ranger-admin:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task2dz75L/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 17:15:11.329700 21722 mini_ranger.cc:193] Using host: 127.21.54.148
I20260811 17:15:11.334611 21722 mini_ranger.cc:227] Ranger admin URL: http://127.21.54.148:50489
Aug 11, 2026 5:15:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:15:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:15:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 5:15:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 5:15:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 5:15:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 5:15:12 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 17:15:12.564224 21722 mini_ranger.cc:160] Time spent starting Ranger: real 9.563s user 0.000s sys 0.013s
Aug 11, 2026 5:15:34 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 5:15:34 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 5:15:34 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 5:15:34 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 5:15:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 5:15:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 5:15:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 5:15:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 5:15:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 5:15:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 5:15:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 5:15:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 5:15:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 5:15:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 5:15:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 5:15:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 5:15:34 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 5:15:35 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 17:15:35.644085 21722 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.54.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.54.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.54.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/rangerkms_127.21.54.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.54.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/rangerkms_127.21.54.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.54.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.54.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.54.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/HTTP_127.21.54.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.54.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/HTTP_127.21.54.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 17:15:35 dist-test-slave-fd7l krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468535, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 17:15:35.746994 21722 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 17:15:35.758708 21722 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 17:15:36.783188 21722 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 17:15:36.783291 21722 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ranger-kms
I20260811 17:15:36.783393 21722 mini_ranger_kms.cc:227] Using postgres at 127.21.54.148:33987
2026-08-11 17:15:37,207 [I] DB FLAVOR :POSTGRES
2026-08-11 17:15:37,208 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 17:15:37,208 [I] Checking connection
2026-08-11 17:15:37,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 17:15:37,556 [I] connection success
2026-08-11 17:15:37,556 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 17:15:37,557 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 17:15:37,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 17:15:37,838 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 17:15:37,838 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 17:15:37,838 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 17:15:37,838 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:33987/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 17:15:38,171 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 17:15:38.698729 21722 mini_ranger_kms.cc:310] Created kms service
I20260811 17:15:38.944842 21722 mini_ranger_kms.cc:326] Created kudu user
I20260811 17:15:38.995496 21722 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 17:15:38 dist-test-slave-fd7l krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468535, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
I20260811 17:15:39.449275 21722 mini_ranger_kms.cc:384] Added ranger policy
I20260811 17:15:39.449437 21722 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ranger-kms:/tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 17:15:39.449458 21722 mini_ranger_kms.cc:240] Using host: 127.21.54.148
I20260811 17:15:39.455112 21722 mini_ranger_kms.cc:276] Ranger KMS PID: 2426
I20260811 17:15:39.455263 21722 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.54.148:36769
Aug 11, 2026 5:15:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:15:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:15:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 5:15:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 5:15:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 5:15:41 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 17:15:42.025353 21722 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.278s user 0.008s sys 0.387s
I20260811 17:15:42.025499 21722 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 17:15:42.025624 21722 mini_ranger_kms.cc:401] 127.21.54.148:36769/kms/v1/keys
Aug 11 17:15:42 dist-test-slave-fd7l krb5kdc[1974](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786468542, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.54.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 17:15:42 dist-test-slave-fd7l krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468542, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.54.148@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
Aug 11 17:15:43 dist-test-slave-fd7l krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468542, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.54.148@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
Aug 11 17:15:45 dist-test-slave-fd7l krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:15:45.890014 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:43037
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:35377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:43037
--ranger_config_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.54.148:36769/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:15:46.055471 2481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:15:46.055802 2481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:15:46.055881 2481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:15:46.060304 2481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:15:46.060405 2481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:15:46.060442 2481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:15:46.060493 2481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:15:46.060568 2481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:15:46.066538 2481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:35377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.54.148:36769/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:43037
--ranger_config_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:43037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:15:46.067808 2481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:15:46.068931 2481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:15:46.079955 2488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:15:46.079926 2491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:15:46.080084 2481 server_base.cc:1034] running on GCE node
W20260811 17:15:46.079989 2489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:15:46.080974 2481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:15:46.082231 2481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:15:46.083405 2481 hybrid_clock.cc:648] HybridClock initialized: now 1786468546083385 us; error 38 us; skew 500 ppm
Aug 11 17:15:46 dist-test-slave-fd7l krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468546, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:15:46.091917 2481 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:15:46.093144 2481 webserver.cc:460] Webserver started at http://127.21.54.190:34847/ using document root <none> and password file <none>
I20260811 17:15:46.093761 2481 fs_manager.cc:359] Metadata directory not provided
I20260811 17:15:46.093827 2481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:15:46.094069 2481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 17:15:46 dist-test-slave-fd7l krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468546, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
I20260811 17:15:46.158841 2481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "ac3f0848fb41474bad5d3e64422b12bb"
format_stamp: "Formatted at 2026-08-11 17:15:46 on dist-test-slave-fd7l"
server_key: "0efb5fb835cf3d4666a18037f1a691a1"
server_key_iv: "528f70a9c5c13261db891e20a9a2f5b8"
server_key_version: "kuduclusterkey@0"
I20260811 17:15:46.159526 2481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "ac3f0848fb41474bad5d3e64422b12bb"
format_stamp: "Formatted at 2026-08-11 17:15:46 on dist-test-slave-fd7l"
server_key: "0efb5fb835cf3d4666a18037f1a691a1"
server_key_iv: "528f70a9c5c13261db891e20a9a2f5b8"
server_key_version: "kuduclusterkey@0"
I20260811 17:15:46.164080 2481 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260811 17:15:46.183575 2499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:15:46.185075 2481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 17:15:46.185228 2481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "ac3f0848fb41474bad5d3e64422b12bb"
format_stamp: "Formatted at 2026-08-11 17:15:46 on dist-test-slave-fd7l"
server_key: "0efb5fb835cf3d4666a18037f1a691a1"
server_key_iv: "528f70a9c5c13261db891e20a9a2f5b8"
server_key_version: "kuduclusterkey@0"
I20260811 17:15:46.185384 2481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:15:46.218226 2481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:15:46.221484 2481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:15:46.221715 2481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:15:46.235428 2481 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:43037
I20260811 17:15:46.235514 2559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:43037 every 8 connection(s)
I20260811 17:15:46.236846 2481 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:15:46.240334 2560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:15:46.241883 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 2481
I20260811 17:15:46.242014 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:15:46.242383 21722 external_mini_cluster.cc:1428] Setting key 24d175921fe5176c4c8baa1ddb8cbb8b
I20260811 17:15:46.250090 2560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb: Bootstrap starting.
I20260811 17:15:46.253800 2560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb: Neither blocks nor log segments found. Creating new log.
I20260811 17:15:46.255857 2560 log.cc:824] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb: Log is configured to *not* fsync() on all Append() calls
Aug 11 17:15:46 dist-test-slave-fd7l krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:15:46.259068 2560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb: No bootstrap required, opened a new log
I20260811 17:15:46.265467 2563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:46.244244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36276 (local address 127.21.54.190:43037)
0811 17:15:46.244938 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0811 17:15:46.244989 (+ 51us) server_negotiation.cc:399] Waiting for connection header
0811 17:15:46.245061 (+ 72us) server_negotiation.cc:407] Connection header received
0811 17:15:46.246490 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:15:46.246529 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:15:46.247017 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:15:46.247515 (+ 498us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:15:46.248627 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:46.250534 (+ 1907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:46.251973 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:46.252867 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:46.253052 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:15:46.257397 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:15:46.257406 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:15:46.257430 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:15:46.257467 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:15:46.260844 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:15:46.261508 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:15:46.261516 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:15:46.261524 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:15:46.261685 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:15:46.262059 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:15:46.262063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:15:46.262076 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:15:46.262831 (+ 755us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:15:46.263056 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 17:15:46.263458 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 17:15:46.264092 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":227,"threads_started":1}
I20260811 17:15:46.265486 2560 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3f0848fb41474bad5d3e64422b12bb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43037 } }
I20260811 17:15:46.265902 2560 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:15:46.266008 2560 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac3f0848fb41474bad5d3e64422b12bb, State: Initialized, Role: FOLLOWER
I20260811 17:15:46.266523 2560 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3f0848fb41474bad5d3e64422b12bb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43037 } }
I20260811 17:15:46.266705 2560 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:15:46.266819 2560 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:15:46.266966 2560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:15:46.268873 2560 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3f0848fb41474bad5d3e64422b12bb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43037 } }
I20260811 17:15:46.269285 2560 leader_election.cc:302] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac3f0848fb41474bad5d3e64422b12bb; no voters:
I20260811 17:15:46.269682 2560 leader_election.cc:288] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:15:46.269874 2566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:15:46.270237 2566 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb [term 1 LEADER]: Becoming Leader. State: Replica: ac3f0848fb41474bad5d3e64422b12bb, State: Running, Role: LEADER
I20260811 17:15:46.270638 2566 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3f0848fb41474bad5d3e64422b12bb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43037 } }
I20260811 17:15:46.271256 2560 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:15:46.272092 2566 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac3f0848fb41474bad5d3e64422b12bb. Latest consensus state: current_term: 1 leader_uuid: "ac3f0848fb41474bad5d3e64422b12bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3f0848fb41474bad5d3e64422b12bb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43037 } } }
I20260811 17:15:46.272308 2566 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb [sys.catalog]: This master's current role is: LEADER
I20260811 17:15:46.273774 2568 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac3f0848fb41474bad5d3e64422b12bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3f0848fb41474bad5d3e64422b12bb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43037 } } }
I20260811 17:15:46.273897 2568 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb [sys.catalog]: This master's current role is: LEADER
I20260811 17:15:46.274348 2576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:15:46.279860 2576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:15:46.280400 2560 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 17:15:46.293953 2576 catalog_manager.cc:1347] Generated new cluster ID: 246840bd914f4bc89c2b4bb65cf23a46
I20260811 17:15:46.294072 2576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:15:46.304690 2576 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:15:46.306270 2576 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:15:46.318739 2576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb: Generated new TSK 0
I20260811 17:15:46.319832 2576 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
I20260811 17:15:47.281148 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.54.148:36769/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:43037
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:35377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:15:47.448640 2612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:15:47.448971 2612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:15:47.449044 2612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:15:47.453826 2612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:15:47.453918 2612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:15:47.454028 2612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:15:47.460758 2612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:35377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.54.148:36769/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:43037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:15:47.462175 2612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:15:47.463773 2612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:15:47.478957 2618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:15:47.479177 2617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:15:47.479714 2612 server_base.cc:1034] running on GCE node
W20260811 17:15:47.479909 2620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:15:47.480492 2612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:15:47.481503 2612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:15:47.482745 2612 hybrid_clock.cc:648] HybridClock initialized: now 1786468547482703 us; error 49 us; skew 500 ppm
Aug 11 17:15:47 dist-test-slave-fd7l krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468547, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:15:47.487771 2612 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:15:47.489169 2612 webserver.cc:460] Webserver started at http://127.21.54.129:40775/ using document root <none> and password file <none>
I20260811 17:15:47.489953 2612 fs_manager.cc:359] Metadata directory not provided
I20260811 17:15:47.490065 2612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:15:47.490370 2612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 17:15:47 dist-test-slave-fd7l krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468547, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
I20260811 17:15:47.509404 2612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "f226e42adfea4036b00bb387828197dd"
format_stamp: "Formatted at 2026-08-11 17:15:47 on dist-test-slave-fd7l"
server_key: "d57c4860aa6c6124f88f0bac08ccaa55"
server_key_iv: "31ff1270ad7cf53bd9790ea8010091a3"
server_key_version: "kuduclusterkey@0"
I20260811 17:15:47.510505 2612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "f226e42adfea4036b00bb387828197dd"
format_stamp: "Formatted at 2026-08-11 17:15:47 on dist-test-slave-fd7l"
server_key: "d57c4860aa6c6124f88f0bac08ccaa55"
server_key_iv: "31ff1270ad7cf53bd9790ea8010091a3"
server_key_version: "kuduclusterkey@0"
I20260811 17:15:47.516464 2612 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 17:15:47.541109 2631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:15:47.543185 2612 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.004s sys 0.000s
I20260811 17:15:47.543460 2612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "f226e42adfea4036b00bb387828197dd"
format_stamp: "Formatted at 2026-08-11 17:15:47 on dist-test-slave-fd7l"
server_key: "d57c4860aa6c6124f88f0bac08ccaa55"
server_key_iv: "31ff1270ad7cf53bd9790ea8010091a3"
server_key_version: "kuduclusterkey@0"
I20260811 17:15:47.543763 2612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:15:47.595743 2612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:15:47.600638 2612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:15:47.600955 2612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:15:47.601881 2612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:15:47.603305 2612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:15:47.603427 2612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 17:15:47.603608 2612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:15:47.603724 2612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:15:47.623865 2612 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:42451
I20260811 17:15:47.625478 2612 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:15:47.627112 2760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:42451 every 8 connection(s)
I20260811 17:15:47.633315 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 2612
I20260811 17:15:47.633460 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:15:47.633848 21722 external_mini_cluster.cc:1428] Setting key ff56624a80464b0ed2a5218622e6807f
Aug 11 17:15:47 dist-test-slave-fd7l krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468547, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:15:47.647182 2765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:47.630112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:41721 (local address 127.21.54.190:43037)
0811 17:15:47.632741 (+ 2629us) server_negotiation.cc:206] Beginning negotiation
0811 17:15:47.632747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:15:47.632762 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:15:47.632811 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:15:47.632814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:15:47.632871 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:15:47.632966 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:15:47.635006 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:47.635865 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:47.637742 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:47.638113 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:47.638195 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:15:47.642016 (+ 3821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:15:47.642020 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:15:47.642023 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:15:47.642057 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:15:47.644319 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:15:47.645036 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:15:47.645039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:15:47.645041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:15:47.645138 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:15:47.645564 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:15:47.645567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:15:47.645570 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:15:47.645843 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:15:47.645933 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 17:15:47.646831 (+ 898us) server_negotiation.cc:299] Negotiation successful
0811 17:15:47.646984 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2539,"thread_start_us":112,"threads_started":1}
I20260811 17:15:47.649000 2764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:47.628760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43037 (local address 127.21.54.129:41721)
0811 17:15:47.630747 (+ 1987us) negotiation.cc:107] Waiting for socket to connect
0811 17:15:47.630808 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 17:15:47.632252 (+ 1444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:15:47.633303 (+ 1051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:15:47.633313 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:15:47.633942 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:15:47.634807 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:15:47.634829 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:15:47.636048 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:15:47.636051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:15:47.637580 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:15:47.637591 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:15:47.638416 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:15:47.638421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:15:47.638712 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:15:47.639579 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:15:47.639610 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:15:47.641793 (+ 2183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:15:47.644484 (+ 2691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:15:47.644491 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:15:47.644529 (+ 38us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:15:47.644909 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:15:47.645246 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:15:47.645249 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:15:47.645251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:15:47.645442 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:15:47.646004 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:15:47.646011 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:15:47.646443 (+ 432us) client_negotiation.cc:769] Sending connection context
0811 17:15:47.647451 (+ 1008us) client_negotiation.cc:240] Negotiation successful
0811 17:15:47.647865 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1691,"thread_start_us":231,"threads_started":1}
I20260811 17:15:47.650480 2761 heartbeater.cc:347] Connected to a master server at 127.21.54.190:43037
I20260811 17:15:47.650745 2761 heartbeater.cc:464] Registering TS with master...
I20260811 17:15:47.651340 2761 heartbeater.cc:511] Master 127.21.54.190:43037 requested a full tablet report, sending...
I20260811 17:15:47.653219 2517 ts_manager.cc:194] Registered new tserver with Master: f226e42adfea4036b00bb387828197dd (127.21.54.129:42451)
I20260811 17:15:47.654822 2517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:41721
WARNING: no policy specified for kudu/127.21.54.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
I20260811 17:15:47.712350 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.54.148:36769/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:43037
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:35377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 17:15:47 dist-test-slave-fd7l krb5kdc[1974](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786468547, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 17:15:47.945130 2770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:15:47.945638 2770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:15:47.945734 2770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:15:47.952303 2770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:15:47.952410 2770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:15:47.952546 2770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:15:47.961532 2770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:35377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.54.148:36769/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:43037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:15:47.963160 2770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:15:47.964725 2770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:15:47.980334 2780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:15:47.980644 2778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:15:47.980819 2777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:15:47.981032 2770 server_base.cc:1034] running on GCE node
I20260811 17:15:47.981767 2770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:15:47.982892 2770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:15:47.986253 2770 hybrid_clock.cc:648] HybridClock initialized: now 1786468547986157 us; error 130 us; skew 500 ppm
Aug 11 17:15:47 dist-test-slave-fd7l krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468547, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:15:47.991851 2770 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:15:47.993186 2770 webserver.cc:460] Webserver started at http://127.21.54.130:35657/ using document root <none> and password file <none>
I20260811 17:15:47.993937 2770 fs_manager.cc:359] Metadata directory not provided
I20260811 17:15:47.994031 2770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:15:47.994287 2770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 17:15:47 dist-test-slave-fd7l krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468547, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
I20260811 17:15:48.017304 2770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "7b567c4b18ce41c5a34179d9adae808a"
format_stamp: "Formatted at 2026-08-11 17:15:48 on dist-test-slave-fd7l"
server_key: "605eec1cb957c6ee059c01354b2926b2"
server_key_iv: "33380ca816f077698b29da51b0546a74"
server_key_version: "kuduclusterkey@0"
I20260811 17:15:48.018755 2770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "7b567c4b18ce41c5a34179d9adae808a"
format_stamp: "Formatted at 2026-08-11 17:15:48 on dist-test-slave-fd7l"
server_key: "605eec1cb957c6ee059c01354b2926b2"
server_key_iv: "33380ca816f077698b29da51b0546a74"
server_key_version: "kuduclusterkey@0"
I20260811 17:15:48.026468 2770 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.004s
I20260811 17:15:48.049944 2787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:15:48.051980 2770 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260811 17:15:48.052193 2770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "7b567c4b18ce41c5a34179d9adae808a"
format_stamp: "Formatted at 2026-08-11 17:15:48 on dist-test-slave-fd7l"
server_key: "605eec1cb957c6ee059c01354b2926b2"
server_key_iv: "33380ca816f077698b29da51b0546a74"
server_key_version: "kuduclusterkey@0"
I20260811 17:15:48.052424 2770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:15:48.089625 2770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:15:48.094048 2770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:15:48.094348 2770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:15:48.095269 2770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:15:48.096777 2770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:15:48.096870 2770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:15:48.097110 2770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:15:48.097168 2770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:15:48.121264 2770 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:44525
I20260811 17:15:48.122864 2770 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:15:48.124811 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 2770
I20260811 17:15:48.124946 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:15:48.125265 21722 external_mini_cluster.cc:1428] Setting key 4a74c636937decc42fb62b1f61030c98
I20260811 17:15:48.125358 2919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:44525 every 8 connection(s)
Aug 11 17:15:48 dist-test-slave-fd7l krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468547, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:15:48.154460 2765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:48.128701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:45207 (local address 127.21.54.190:43037)
0811 17:15:48.128883 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 17:15:48.128888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:15:48.130676 (+ 1788us) server_negotiation.cc:407] Connection header received
0811 17:15:48.132423 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:15:48.132426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:15:48.132486 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:15:48.132562 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:15:48.137785 (+ 5223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.138631 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.140812 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.141241 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.141338 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:15:48.149279 (+ 7941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:15:48.149283 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:15:48.149287 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:15:48.149324 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:15:48.151173 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:15:48.151857 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:15:48.151861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:15:48.151863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:15:48.151961 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:15:48.153153 (+ 1192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:15:48.153156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:15:48.153157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:15:48.153357 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:15:48.153487 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 17:15:48.154105 (+ 618us) server_negotiation.cc:299] Negotiation successful
0811 17:15:48.154256 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
WARNING: no policy specified for kudu/127.21.54.131@KRBTEST.COM; defaulting to no policy
I20260811 17:15:48.155465 2922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:48.127878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43037 (local address 127.21.54.130:45207)
0811 17:15:48.130518 (+ 2640us) negotiation.cc:107] Waiting for socket to connect
0811 17:15:48.130572 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 17:15:48.132115 (+ 1543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:15:48.135869 (+ 3754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:15:48.135882 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:15:48.136571 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:15:48.137533 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:15:48.137560 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.138855 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.138859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:15:48.140643 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:15:48.140655 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.141626 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.141642 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:15:48.141893 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:15:48.142993 (+ 1100us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:15:48.143029 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:15:48.148965 (+ 5936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:15:48.151330 (+ 2365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:15:48.151338 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:15:48.151359 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:15:48.151741 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:15:48.152834 (+ 1093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:15:48.152837 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:15:48.152838 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:15:48.153025 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:15:48.153490 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:15:48.153501 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:15:48.153807 (+ 306us) client_negotiation.cc:769] Sending connection context
0811 17:15:48.154046 (+ 239us) client_negotiation.cc:240] Negotiation successful
0811 17:15:48.154414 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2238,"thread_start_us":266,"threads_started":1}
I20260811 17:15:48.157449 2920 heartbeater.cc:347] Connected to a master server at 127.21.54.190:43037
I20260811 17:15:48.157749 2920 heartbeater.cc:464] Registering TS with master...
I20260811 17:15:48.158398 2920 heartbeater.cc:511] Master 127.21.54.190:43037 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.131@KRBTEST.COM" created.
I20260811 17:15:48.159869 2517 ts_manager.cc:194] Registered new tserver with Master: 7b567c4b18ce41c5a34179d9adae808a (127.21.54.130:44525)
I20260811 17:15:48.160787 2517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:45207
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
I20260811 17:15:48.185362 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.54.148:36769/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:43037
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:35377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:15:48.382340 2926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:15:48.382769 2926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:15:48.382841 2926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:15:48.389879 2926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:15:48.390044 2926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:15:48.390172 2926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:15:48.398998 2926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:35377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.54.148:36769/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:43037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:15:48.400545 2926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:15:48.401906 2926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:15:48.418322 2932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:15:48.419898 2933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:15:48.420135 2935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:15:48.420852 2926 server_base.cc:1034] running on GCE node
I20260811 17:15:48.421432 2926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:15:48.422281 2926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
Aug 11 17:15:48 dist-test-slave-fd7l krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468547, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
I20260811 17:15:48.437808 2926 hybrid_clock.cc:648] HybridClock initialized: now 1786468548437684 us; error 167 us; skew 500 ppm
Aug 11 17:15:48 dist-test-slave-fd7l krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468548, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:15:48.442869 2926 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:15:48.444213 2926 webserver.cc:460] Webserver started at http://127.21.54.131:35815/ using document root <none> and password file <none>
I20260811 17:15:48.445075 2926 fs_manager.cc:359] Metadata directory not provided
I20260811 17:15:48.445179 2926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:15:48.445485 2926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 17:15:48 dist-test-slave-fd7l krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468548, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
I20260811 17:15:48.464210 2926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "49831da567234c8b838ee17d5ba5fdad"
format_stamp: "Formatted at 2026-08-11 17:15:48 on dist-test-slave-fd7l"
server_key: "1ca7f7422c1be34165d0e40fe3082e04"
server_key_iv: "3817a0edd34f899de150aee322e19a3e"
server_key_version: "kuduclusterkey@0"
I20260811 17:15:48.465126 2926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "49831da567234c8b838ee17d5ba5fdad"
format_stamp: "Formatted at 2026-08-11 17:15:48 on dist-test-slave-fd7l"
server_key: "1ca7f7422c1be34165d0e40fe3082e04"
server_key_iv: "3817a0edd34f899de150aee322e19a3e"
server_key_version: "kuduclusterkey@0"
I20260811 17:15:48.471441 2926 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.002s
I20260811 17:15:48.496682 2944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:15:48.510295 2926 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.002s sys 0.001s
I20260811 17:15:48.510493 2926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "49831da567234c8b838ee17d5ba5fdad"
format_stamp: "Formatted at 2026-08-11 17:15:48 on dist-test-slave-fd7l"
server_key: "1ca7f7422c1be34165d0e40fe3082e04"
server_key_iv: "3817a0edd34f899de150aee322e19a3e"
server_key_version: "kuduclusterkey@0"
I20260811 17:15:48.510680 2926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:15:48.546535 2926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:15:48.563082 2926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:15:48.563323 2926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:15:48.564316 2926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:15:48.565724 2926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:15:48.565795 2926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:15:48.565855 2926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:15:48.565887 2926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:15:48.591420 2926 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:37053
I20260811 17:15:48.591559 3079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:37053 every 8 connection(s)
I20260811 17:15:48.593792 2926 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:15:48.601071 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 2926
I20260811 17:15:48.601217 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:15:48.601599 21722 external_mini_cluster.cc:1428] Setting key 368ddd680631c96b4fface25c922042e
Aug 11 17:15:48 dist-test-slave-fd7l krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468548, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:15:48.615557 2765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:48.596795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:40077 (local address 127.21.54.190:43037)
0811 17:15:48.596953 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 17:15:48.596958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:15:48.598829 (+ 1871us) server_negotiation.cc:407] Connection header received
0811 17:15:48.600754 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:15:48.600768 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:15:48.600856 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:15:48.601006 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:15:48.603591 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.604862 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.606784 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.607323 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.607547 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:15:48.611013 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:15:48.611017 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:15:48.611020 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:15:48.611054 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:15:48.612944 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:15:48.613557 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:15:48.613560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:15:48.613562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:15:48.613656 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:15:48.614114 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:15:48.614118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:15:48.614119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:15:48.614345 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:15:48.614418 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 17:15:48.615238 (+ 820us) server_negotiation.cc:299] Negotiation successful
0811 17:15:48.615372 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 17:15:48.617245 3082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:48.597734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43037 (local address 127.21.54.131:40077)
0811 17:15:48.598639 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0811 17:15:48.598703 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 17:15:48.600331 (+ 1628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:15:48.601531 (+ 1200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:15:48.601550 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:15:48.602174 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:15:48.603287 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:15:48.603318 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.605039 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.605042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:15:48.606616 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:15:48.606628 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.607606 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.607610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:15:48.607876 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:15:48.608899 (+ 1023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:15:48.608921 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:15:48.610801 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:15:48.613076 (+ 2275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:15:48.613083 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:15:48.613100 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:15:48.613442 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:15:48.613760 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:15:48.613763 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:15:48.613765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:15:48.613981 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:15:48.614560 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:15:48.614566 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:15:48.614928 (+ 362us) client_negotiation.cc:769] Sending connection context
0811 17:15:48.615775 (+ 847us) client_negotiation.cc:240] Negotiation successful
0811 17:15:48.616125 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":277,"threads_started":1}
I20260811 17:15:48.618613 3080 heartbeater.cc:347] Connected to a master server at 127.21.54.190:43037
I20260811 17:15:48.618870 3080 heartbeater.cc:464] Registering TS with master...
I20260811 17:15:48.619431 3080 heartbeater.cc:511] Master 127.21.54.190:43037 requested a full tablet report, sending...
I20260811 17:15:48.620616 2517 ts_manager.cc:194] Registered new tserver with Master: 49831da567234c8b838ee17d5ba5fdad (127.21.54.131:37053)
I20260811 17:15:48.621387 2517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:40077
I20260811 17:15:48.628206 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:15:48.643044 2765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:48.632175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36286 (local address 127.21.54.190:43037)
0811 17:15:48.632323 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 17:15:48.632328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:15:48.632876 (+ 548us) server_negotiation.cc:407] Connection header received
0811 17:15:48.633014 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:15:48.633017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:15:48.633073 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:15:48.633165 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:15:48.634243 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.635085 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.636301 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.636687 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.636797 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:15:48.638911 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:15:48.638915 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:15:48.638918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:15:48.638977 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:15:48.640967 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:15:48.641521 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:15:48.641524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:15:48.641526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:15:48.641627 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:15:48.641967 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:15:48.641971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:15:48.641972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:15:48.642228 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:15:48.642361 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 17:15:48.642680 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 17:15:48.642864 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 17:15:48.647136 2517 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:15:48.651960 2517 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:15:48.657997 2761 heartbeater.cc:503] Master 127.21.54.190:43037 was elected leader, sending a full tablet report...
I20260811 17:15:48.672967 3098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:48.664866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:37053 (local address 127.0.0.1:33040)
0811 17:15:48.665838 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0811 17:15:48.665892 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 17:15:48.666044 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:15:48.667162 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:15:48.667168 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:15:48.667191 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:15:48.667316 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:15:48.667322 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.668994 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.668998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:15:48.670508 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:15:48.670520 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.672252 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.672256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:15:48.672392 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:15:48.672436 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 17:15:48.672578 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 17:15:48.672768 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"thread_start_us":184,"threads_started":1}
I20260811 17:15:48.673921 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:48.665003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33040 (local address 127.21.54.131:37053)
0811 17:15:48.666439 (+ 1436us) server_negotiation.cc:206] Beginning negotiation
0811 17:15:48.666482 (+ 43us) server_negotiation.cc:399] Waiting for connection header
0811 17:15:48.666509 (+ 27us) server_negotiation.cc:407] Connection header received
0811 17:15:48.666586 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:15:48.666591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:15:48.666798 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:15:48.666990 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:15:48.667442 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.668840 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.670756 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.671832 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.672004 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:15:48.672183 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 17:15:48.673548 (+ 1365us) server_negotiation.cc:299] Negotiation successful
0811 17:15:48.673714 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1321,"thread_start_us":124,"threads_started":1}
I20260811 17:15:48.678189 2993 tablet_service.cc:1467] Processing CreateTablet for tablet 94aa80d44db840bea8ee1eab2e176141 (DEFAULT_TABLE table=test-table [id=28d477a152a74dc1817eaf3abdd4738c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:15:48.680393 2993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94aa80d44db840bea8ee1eab2e176141. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:15:48.683044 3103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:48.665747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:42451 (local address 127.0.0.1:46366)
0811 17:15:48.667714 (+ 1967us) negotiation.cc:107] Waiting for socket to connect
0811 17:15:48.667728 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:15:48.667849 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:15:48.670505 (+ 2656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:15:48.670517 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:15:48.670547 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:15:48.670660 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:15:48.670666 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.675172 (+ 4506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.675177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:15:48.676731 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:15:48.676743 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.682690 (+ 5947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.682693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:15:48.682768 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:15:48.682794 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 17:15:48.682842 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 17:15:48.682906 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1893,"thread_start_us":156,"threads_started":1}
I20260811 17:15:48.683765 3101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:48.665899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46366 (local address 127.21.54.129:42451)
0811 17:15:48.668137 (+ 2238us) server_negotiation.cc:206] Beginning negotiation
0811 17:15:48.668143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:15:48.668167 (+ 24us) server_negotiation.cc:407] Connection header received
0811 17:15:48.668242 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:15:48.668248 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:15:48.668464 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:15:48.670828 (+ 2364us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:15:48.671034 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.672158 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.676936 (+ 4778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.678734 (+ 1798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.683373 (+ 4639us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:15:48.683518 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 17:15:48.683566 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 17:15:48.683653 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2130,"thread_start_us":84,"threads_started":1}
I20260811 17:15:48.685217 3100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:48.666804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:44525 (local address 127.0.0.1:37498)
0811 17:15:48.668736 (+ 1932us) negotiation.cc:107] Waiting for socket to connect
0811 17:15:48.668750 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:15:48.668872 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:15:48.672538 (+ 3666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:15:48.672541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:15:48.672560 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:15:48.672656 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:15:48.672662 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.679104 (+ 6442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.679107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:15:48.680545 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:15:48.680559 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.684729 (+ 4170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.684733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:15:48.684878 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:15:48.684913 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 17:15:48.685005 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 17:15:48.685090 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1850,"thread_start_us":143,"threads_started":1}
I20260811 17:15:48.685321 3102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:48.666500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37498 (local address 127.21.54.130:44525)
0811 17:15:48.669121 (+ 2621us) server_negotiation.cc:206] Beginning negotiation
0811 17:15:48.669129 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:15:48.669153 (+ 24us) server_negotiation.cc:407] Connection header received
0811 17:15:48.669232 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:15:48.669237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:15:48.669441 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:15:48.672369 (+ 2928us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:15:48.672795 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.678957 (+ 6162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.680710 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.681731 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.684702 (+ 2971us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:15:48.684880 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 17:15:48.685058 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 17:15:48.685168 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2516,"thread_start_us":92,"threads_started":1}
I20260811 17:15:48.688005 2687 tablet_service.cc:1467] Processing CreateTablet for tablet 94aa80d44db840bea8ee1eab2e176141 (DEFAULT_TABLE table=test-table [id=28d477a152a74dc1817eaf3abdd4738c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:15:48.689322 2687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94aa80d44db840bea8ee1eab2e176141. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:15:48.690696 2842 tablet_service.cc:1467] Processing CreateTablet for tablet 94aa80d44db840bea8ee1eab2e176141 (DEFAULT_TABLE table=test-table [id=28d477a152a74dc1817eaf3abdd4738c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:15:48.692116 2842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94aa80d44db840bea8ee1eab2e176141. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:15:48.704548 3104 tablet_bootstrap.cc:492] T 94aa80d44db840bea8ee1eab2e176141 P f226e42adfea4036b00bb387828197dd: Bootstrap starting.
I20260811 17:15:48.708166 3105 tablet_bootstrap.cc:492] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a: Bootstrap starting.
I20260811 17:15:48.710806 3105 tablet_bootstrap.cc:654] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a: Neither blocks nor log segments found. Creating new log.
I20260811 17:15:48.711985 3105 log.cc:824] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a: Log is configured to *not* fsync() on all Append() calls
I20260811 17:15:48.712967 3106 tablet_bootstrap.cc:492] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad: Bootstrap starting.
I20260811 17:15:48.713881 3104 tablet_bootstrap.cc:654] T 94aa80d44db840bea8ee1eab2e176141 P f226e42adfea4036b00bb387828197dd: Neither blocks nor log segments found. Creating new log.
I20260811 17:15:48.715719 3104 log.cc:824] T 94aa80d44db840bea8ee1eab2e176141 P f226e42adfea4036b00bb387828197dd: Log is configured to *not* fsync() on all Append() calls
I20260811 17:15:48.718456 3105 tablet_bootstrap.cc:492] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a: No bootstrap required, opened a new log
I20260811 17:15:48.718786 3105 ts_tablet_manager.cc:1397] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260811 17:15:48.718870 3106 tablet_bootstrap.cc:654] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad: Neither blocks nor log segments found. Creating new log.
I20260811 17:15:48.719076 3104 tablet_bootstrap.cc:492] T 94aa80d44db840bea8ee1eab2e176141 P f226e42adfea4036b00bb387828197dd: No bootstrap required, opened a new log
I20260811 17:15:48.719345 3104 ts_tablet_manager.cc:1397] T 94aa80d44db840bea8ee1eab2e176141 P f226e42adfea4036b00bb387828197dd: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.000s
I20260811 17:15:48.720570 3106 log.cc:824] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad: Log is configured to *not* fsync() on all Append() calls
I20260811 17:15:48.723376 3106 tablet_bootstrap.cc:492] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad: No bootstrap required, opened a new log
I20260811 17:15:48.723740 3106 ts_tablet_manager.cc:1397] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 17:15:48.725725 3105 raft_consensus.cc:348] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49831da567234c8b838ee17d5ba5fdad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37053 } } peers { permanent_uuid: "7b567c4b18ce41c5a34179d9adae808a" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 44525 } } peers { permanent_uuid: "f226e42adfea4036b00bb387828197dd" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42451 } }
I20260811 17:15:48.726037 3105 raft_consensus.cc:374] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:15:48.726095 3105 raft_consensus.cc:729] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b567c4b18ce41c5a34179d9adae808a, State: Initialized, Role: FOLLOWER
I20260811 17:15:48.726420 3104 raft_consensus.cc:348] T 94aa80d44db840bea8ee1eab2e176141 P f226e42adfea4036b00bb387828197dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49831da567234c8b838ee17d5ba5fdad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37053 } } peers { permanent_uuid: "7b567c4b18ce41c5a34179d9adae808a" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 44525 } } peers { permanent_uuid: "f226e42adfea4036b00bb387828197dd" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42451 } }
I20260811 17:15:48.726686 3105 consensus_queue.cc:260] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49831da567234c8b838ee17d5ba5fdad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37053 } } peers { permanent_uuid: "7b567c4b18ce41c5a34179d9adae808a" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 44525 } } peers { permanent_uuid: "f226e42adfea4036b00bb387828197dd" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42451 } }
I20260811 17:15:48.727105 3104 raft_consensus.cc:374] T 94aa80d44db840bea8ee1eab2e176141 P f226e42adfea4036b00bb387828197dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:15:48.727289 3104 raft_consensus.cc:729] T 94aa80d44db840bea8ee1eab2e176141 P f226e42adfea4036b00bb387828197dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f226e42adfea4036b00bb387828197dd, State: Initialized, Role: FOLLOWER
I20260811 17:15:48.728019 3105 ts_tablet_manager.cc:1428] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a: Time spent starting tablet: real 0.009s user 0.006s sys 0.004s
I20260811 17:15:48.728428 2920 heartbeater.cc:503] Master 127.21.54.190:43037 was elected leader, sending a full tablet report...
I20260811 17:15:48.728421 3104 consensus_queue.cc:260] T 94aa80d44db840bea8ee1eab2e176141 P f226e42adfea4036b00bb387828197dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49831da567234c8b838ee17d5ba5fdad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37053 } } peers { permanent_uuid: "7b567c4b18ce41c5a34179d9adae808a" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 44525 } } peers { permanent_uuid: "f226e42adfea4036b00bb387828197dd" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42451 } }
I20260811 17:15:48.729910 3104 ts_tablet_manager.cc:1428] T 94aa80d44db840bea8ee1eab2e176141 P f226e42adfea4036b00bb387828197dd: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260811 17:15:48.732810 3106 raft_consensus.cc:348] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49831da567234c8b838ee17d5ba5fdad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37053 } } peers { permanent_uuid: "7b567c4b18ce41c5a34179d9adae808a" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 44525 } } peers { permanent_uuid: "f226e42adfea4036b00bb387828197dd" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42451 } }
I20260811 17:15:48.733323 3106 raft_consensus.cc:374] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:15:48.733441 3106 raft_consensus.cc:729] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49831da567234c8b838ee17d5ba5fdad, State: Initialized, Role: FOLLOWER
I20260811 17:15:48.733991 3106 consensus_queue.cc:260] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49831da567234c8b838ee17d5ba5fdad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37053 } } peers { permanent_uuid: "7b567c4b18ce41c5a34179d9adae808a" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 44525 } } peers { permanent_uuid: "f226e42adfea4036b00bb387828197dd" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42451 } }
I20260811 17:15:48.735270 3106 ts_tablet_manager.cc:1428] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad: Time spent starting tablet: real 0.011s user 0.005s sys 0.006s
I20260811 17:15:48.735702 3080 heartbeater.cc:503] Master 127.21.54.190:43037 was elected leader, sending a full tablet report...
Aug 11 17:15:48 dist-test-slave-fd7l krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468547, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
W20260811 17:15:48.846751 3081 tablet.cc:2369] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:15:48.850251 3110 raft_consensus.cc:482] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:15:48.850540 3110 raft_consensus.cc:504] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49831da567234c8b838ee17d5ba5fdad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37053 } } peers { permanent_uuid: "7b567c4b18ce41c5a34179d9adae808a" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 44525 } } peers { permanent_uuid: "f226e42adfea4036b00bb387828197dd" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42451 } }
I20260811 17:15:48.851938 3110 leader_election.cc:288] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49831da567234c8b838ee17d5ba5fdad (127.21.54.131:37053), f226e42adfea4036b00bb387828197dd (127.21.54.129:42451)
I20260811 17:15:48.857975 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:48.853418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:59199 (local address 127.21.54.131:37053)
0811 17:15:48.853572 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 17:15:48.853577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:15:48.853595 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:15:48.853655 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:15:48.853658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:15:48.853712 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:15:48.853791 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:15:48.854286 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.854799 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.856754 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.857252 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.857401 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:15:48.857482 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 17:15:48.857789 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 17:15:48.857874 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 17:15:48.858047 3114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:48.852473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:37053 (local address 127.21.54.130:59199)
0811 17:15:48.852928 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 17:15:48.852943 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:15:48.853094 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:15:48.853827 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:15:48.853830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:15:48.853852 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:15:48.854155 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:15:48.854162 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.854996 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.855001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:15:48.856599 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:15:48.856615 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.857498 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.857505 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:15:48.857631 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:15:48.857654 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 17:15:48.857743 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 17:15:48.857824 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":141,"threads_started":1}
I20260811 17:15:48.859243 3024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94aa80d44db840bea8ee1eab2e176141" candidate_uuid: "7b567c4b18ce41c5a34179d9adae808a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49831da567234c8b838ee17d5ba5fdad" is_pre_election: true
I20260811 17:15:48.859745 3024 raft_consensus.cc:2454] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b567c4b18ce41c5a34179d9adae808a in term 0.
I20260811 17:15:48.860466 2789 leader_election.cc:302] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49831da567234c8b838ee17d5ba5fdad, 7b567c4b18ce41c5a34179d9adae808a; no voters:
I20260811 17:15:48.860824 3110 raft_consensus.cc:2781] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:15:48.860924 3110 raft_consensus.cc:482] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:15:48.860982 3110 raft_consensus.cc:3037] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:15:48.863112 3110 raft_consensus.cc:504] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49831da567234c8b838ee17d5ba5fdad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37053 } } peers { permanent_uuid: "7b567c4b18ce41c5a34179d9adae808a" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 44525 } } peers { permanent_uuid: "f226e42adfea4036b00bb387828197dd" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42451 } }
I20260811 17:15:48.863564 3110 leader_election.cc:288] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a [CANDIDATE]: Term 1 election: Requested vote from peers 49831da567234c8b838ee17d5ba5fdad (127.21.54.131:37053), f226e42adfea4036b00bb387828197dd (127.21.54.129:42451)
I20260811 17:15:48.864113 3024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94aa80d44db840bea8ee1eab2e176141" candidate_uuid: "7b567c4b18ce41c5a34179d9adae808a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49831da567234c8b838ee17d5ba5fdad"
I20260811 17:15:48.864276 3024 raft_consensus.cc:3037] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:15:48.864794 3101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:48.853531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:35523 (local address 127.21.54.129:42451)
0811 17:15:48.853657 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 17:15:48.853662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:15:48.858479 (+ 4817us) server_negotiation.cc:407] Connection header received
0811 17:15:48.859017 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:15:48.859021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:15:48.859075 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:15:48.859136 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:15:48.860107 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.860865 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.863100 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.863682 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.863775 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:15:48.863829 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 17:15:48.864576 (+ 747us) server_negotiation.cc:299] Negotiation successful
0811 17:15:48.864665 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 17:15:48.865542 3115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:48.853016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:42451 (local address 127.21.54.130:35523)
0811 17:15:48.858331 (+ 5315us) negotiation.cc:107] Waiting for socket to connect
0811 17:15:48.858346 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:15:48.858505 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:15:48.859296 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:15:48.859298 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:15:48.859318 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:15:48.859964 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:15:48.859972 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.861439 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.861443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:15:48.862949 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:15:48.862961 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.863926 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.863929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:15:48.864385 (+ 456us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:15:48.864417 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 17:15:48.865263 (+ 846us) client_negotiation.cc:240] Negotiation successful
0811 17:15:48.865350 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5234,"thread_start_us":98,"threads_started":1}
I20260811 17:15:48.866251 3024 raft_consensus.cc:2454] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b567c4b18ce41c5a34179d9adae808a in term 1.
I20260811 17:15:48.866806 2789 leader_election.cc:302] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49831da567234c8b838ee17d5ba5fdad, 7b567c4b18ce41c5a34179d9adae808a; no voters:
I20260811 17:15:48.867543 2693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94aa80d44db840bea8ee1eab2e176141" candidate_uuid: "7b567c4b18ce41c5a34179d9adae808a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f226e42adfea4036b00bb387828197dd" is_pre_election: true
I20260811 17:15:48.867671 3110 raft_consensus.cc:2781] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:15:48.867882 2693 raft_consensus.cc:2454] T 94aa80d44db840bea8ee1eab2e176141 P f226e42adfea4036b00bb387828197dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b567c4b18ce41c5a34179d9adae808a in term 0.
I20260811 17:15:48.868026 3110 raft_consensus.cc:686] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a [term 1 LEADER]: Becoming Leader. State: Replica: 7b567c4b18ce41c5a34179d9adae808a, State: Running, Role: LEADER
I20260811 17:15:48.867506 2694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94aa80d44db840bea8ee1eab2e176141" candidate_uuid: "7b567c4b18ce41c5a34179d9adae808a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f226e42adfea4036b00bb387828197dd"
I20260811 17:15:48.868257 2694 raft_consensus.cc:3037] T 94aa80d44db840bea8ee1eab2e176141 P f226e42adfea4036b00bb387828197dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:15:48.868420 3110 consensus_queue.cc:237] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49831da567234c8b838ee17d5ba5fdad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37053 } } peers { permanent_uuid: "7b567c4b18ce41c5a34179d9adae808a" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 44525 } } peers { permanent_uuid: "f226e42adfea4036b00bb387828197dd" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42451 } }
I20260811 17:15:48.870153 2694 raft_consensus.cc:2454] T 94aa80d44db840bea8ee1eab2e176141 P f226e42adfea4036b00bb387828197dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b567c4b18ce41c5a34179d9adae808a in term 1.
I20260811 17:15:48.873155 2516 catalog_manager.cc:5495] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b567c4b18ce41c5a34179d9adae808a (127.21.54.130). New cstate: current_term: 1 leader_uuid: "7b567c4b18ce41c5a34179d9adae808a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49831da567234c8b838ee17d5ba5fdad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b567c4b18ce41c5a34179d9adae808a" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 44525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f226e42adfea4036b00bb387828197dd" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42451 } health_report { overall_health: UNKNOWN } } }
W20260811 17:15:48.880726 2921 tablet.cc:2369] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:15:48.880792 2762 tablet.cc:2369] T 94aa80d44db840bea8ee1eab2e176141 P f226e42adfea4036b00bb387828197dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:15:48.913555 3102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:48.907973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37502 (local address 127.21.54.130:44525)
0811 17:15:48.908091 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 17:15:48.908111 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 17:15:48.908157 (+ 46us) server_negotiation.cc:407] Connection header received
0811 17:15:48.908378 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:15:48.908382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:15:48.908477 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:15:48.908620 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:15:48.909100 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.909904 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.911245 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:15:48.911600 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:15:48.911708 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:15:48.912182 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:15:48.912809 (+ 627us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:15:48.912950 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 17:15:48.913337 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 17:15:48.913406 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 17:15:48.923841 2694 raft_consensus.cc:1261] T 94aa80d44db840bea8ee1eab2e176141 P f226e42adfea4036b00bb387828197dd [term 1 FOLLOWER]: Refusing update from remote peer 7b567c4b18ce41c5a34179d9adae808a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:15:48.923841 3024 raft_consensus.cc:1261] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad [term 1 FOLLOWER]: Refusing update from remote peer 7b567c4b18ce41c5a34179d9adae808a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:15:48.924901 3116 consensus_queue.cc:1035] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f226e42adfea4036b00bb387828197dd" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:15:48.925454 3116 consensus_queue.cc:1035] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a [LEADER]: Connected to new peer: Peer: permanent_uuid: "49831da567234c8b838ee17d5ba5fdad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:15:48.944301 3121 mvcc.cc:205] Tried to move back new op lower bound from 7317375176381251584 to 7317375176169181184. Current Snapshot: MvccSnapshot[applied={T|T < 7317375176381251584}]
I20260811 17:15:48.948570 3122 mvcc.cc:205] Tried to move back new op lower bound from 7317375176381251584 to 7317375176169181184. Current Snapshot: MvccSnapshot[applied={T|T < 7317375176381251584}]
I20260811 17:15:48.954523 3123 mvcc.cc:205] Tried to move back new op lower bound from 7317375176381251584 to 7317375176169181184. Current Snapshot: MvccSnapshot[applied={T|T < 7317375176381251584}]
I20260811 17:15:48.960716 2516 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:15:48.960916 2516 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:15:48.964041 2516 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac3f0848fb41474bad5d3e64422b12bb: Sending DeleteTablet for 3 replicas of tablet 94aa80d44db840bea8ee1eab2e176141
I20260811 17:15:48.965123 2842 tablet_service.cc:1514] Processing DeleteTablet for tablet 94aa80d44db840bea8ee1eab2e176141 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:15:48 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:37498
I20260811 17:15:48.965193 2993 tablet_service.cc:1514] Processing DeleteTablet for tablet 94aa80d44db840bea8ee1eab2e176141 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:15:48 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:33040
I20260811 17:15:48.965749 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 2612
W20260811 17:15:48.978880 2500 connection.cc:535] client connection to 127.21.54.129:42451 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 17:15:48.979096 2500 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 17:15:48.981247 3100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:48.980711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:42451 (local address 127.0.0.1:46378)
0811 17:15:48.981007 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 17:15:48.981125 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.129:42451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
W20260811 17:15:48.981683 2500 catalog_manager.cc:4584] TS f226e42adfea4036b00bb387828197dd (127.21.54.129:42451): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 94aa80d44db840bea8ee1eab2e176141: Network error: Client connection negotiation failed: client connection to 127.21.54.129:42451: connect: Connection refused (error 111)
I20260811 17:15:48.982314 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 2770
I20260811 17:15:48.982303 3130 tablet_replica.cc:330] T 94aa80d44db840bea8ee1eab2e176141 P 7b567c4b18ce41c5a34179d9adae808a: stopping tablet replica
I20260811 17:15:48.984053 3131 tablet_replica.cc:330] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad: stopping tablet replica
I20260811 17:15:48.984560 3131 raft_consensus.cc:2229] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:15:48.985072 3131 raft_consensus.cc:2258] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:15:48.986825 3131 ts_tablet_manager.cc:1905] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:15:48.990123 3131 ts_tablet_manager.cc:1918] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:15:48.990285 3131 log.cc:1196] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/94aa80d44db840bea8ee1eab2e176141
I20260811 17:15:48.990707 3131 ts_tablet_manager.cc:1939] T 94aa80d44db840bea8ee1eab2e176141 P 49831da567234c8b838ee17d5ba5fdad: Deleting consensus metadata
I20260811 17:15:48.992600 2500 catalog_manager.cc:4841] TS 49831da567234c8b838ee17d5ba5fdad (127.21.54.131:37053): tablet 94aa80d44db840bea8ee1eab2e176141 (table test-table [id=28d477a152a74dc1817eaf3abdd4738c]) successfully deleted
I20260811 17:15:49.004920 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 2926
I20260811 17:15:49.008073 3100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:15:49.005230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:44525 (local address 127.0.0.1:37512)
0811 17:15:49.007853 (+ 2623us) negotiation.cc:107] Waiting for socket to connect
0811 17:15:49.007932 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.130:44525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2495}
I20260811 17:15:49.017426 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 2481
I20260811 17:15:49.029688 21722 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 17:15:49.385099 21722 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 17:15:49.385244 21722 mini_ranger.cc:66] Stopping Ranger...
I20260811 17:15:49.730608 21722 mini_ranger.cc:68] Stopped Ranger
2026-08-11 17:15:49.730 UTC [2012] LOG: received smart shutdown request
2026-08-11 17:15:49.735 UTC [2012] LOG: background worker "logical replication launcher" (PID 2022) exited with exit code 1
2026-08-11 17:15:49.735 UTC [2016] LOG: shutting down
2026-08-11 17:15:49.930 UTC [2012] LOG: database system is shut down
2026-08-11T17:15:49Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52340 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:15:50 dist-test-slave-fd7l krb5kdc[3142](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:15:50 dist-test-slave-fd7l krb5kdc[3142](info): set up 2 sockets
Aug 11 17:15:50 dist-test-slave-fd7l krb5kdc[3142](info): commencing operation
krb5kdc: starting...
W20260811 17:15:51.200911 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:15:51 dist-test-slave-fd7l krb5kdc[3142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:15:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:15:51Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.54.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.54.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.54.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/rangeradmin_127.21.54.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.54.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/rangeradmin_127.21.54.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.54.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.54.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.54.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/rangerlookup_127.21.54.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.54.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/rangerlookup_127.21.54.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.54.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.54.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.54.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/HTTP_127.21.54.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.54.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/HTTP_127.21.54.148@KRBTEST.COM.keytab.
I20260811 17:15:51.397217 21722 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task2dz75L/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/postgres -l logfile start
2026-08-11 17:15:54.196 UTC [3175] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 17:15:54.196 UTC [3175] LOG: listening on IPv4 address "127.21.54.148", port 51495
2026-08-11 17:15:54.200 UTC [3175] LOG: listening on Unix socket "/tmp/.s.PGSQL.51495"
2026-08-11 17:15:54.224 UTC [3178] LOG: database system was shut down at 2026-08-11 17:15:52 UTC
2026-08-11 17:15:54.228 UTC [3175] LOG: database system is ready to accept connections
I20260811 17:15:55.349473 21722 mini_postgres.cc:96] Postgres bound to 51495
2026-08-11 17:15:55.357 UTC [3186] FATAL: database "slave" does not exist
127.21.54.148:51495 - accepting connections
I20260811 17:15:55.358533 21722 mini_ranger.cc:161] Starting Ranger...
I20260811 17:15:55.375749 21722 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 17:15:55.904313 21722 mini_ranger.cc:87] Created ranger Postgres database
I20260811 17:15:55.904407 21722 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ranger-admin
2026-08-11 17:15:56,216 [I] DB FLAVOR :POSTGRES
2026-08-11 17:15:56,217 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 17:15:56,217 [I] Checking connection..
2026-08-11 17:15:56,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 17:15:56,480 [I] Checking connection passed.
2026-08-11 17:15:56,480 [I] --------- Verifying version history table ---------
2026-08-11 17:15:56,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 17:15:56,763 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 17:15:56,763 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 17:15:57,034 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 17:15:57,034 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 17:15:57,034 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 17:15:57.306 UTC [3261] WARNING: there is no transaction in progress
2026-08-11 17:15:57,315 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 17:15:57,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 17:15:57,568 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 17:15:57,569 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 17:15:57,569 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 17:15:57,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 17:15:58,133 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-fd7l.c.gcp-upstream.internal','N') ;"
2026-08-11 17:15:58,390 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 17:15:58,652 [I] Table x_portal_user does not exist in database ranger
2026-08-11 17:15:58,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 17:15:58,916 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 17:15:58,916 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 17:15:59,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 17:15:59,468 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 17:15:59,469 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 17:16:00.522 UTC [3388] WARNING: there is no transaction in progress
2026-08-11 17:16:00.541 UTC [3388] WARNING: there is no transaction in progress
2026-08-11 17:16:00.559 UTC [3388] WARNING: there is no transaction in progress
2026-08-11 17:16:00.576 UTC [3388] WARNING: there is no transaction in progress
2026-08-11 17:16:00.594 UTC [3388] WARNING: there is no transaction in progress
2026-08-11 17:16:00.614 UTC [3388] WARNING: there is no transaction in progress
2026-08-11 17:16:00.675 UTC [3388] WARNING: there is no transaction in progress
2026-08-11 17:16:00.682 UTC [3388] WARNING: there is no transaction in progress
2026-08-11 17:16:00.697 UTC [3388] WARNING: there is no transaction in progress
2026-08-11 17:16:00.708 UTC [3388] WARNING: there is no transaction in progress
2026-08-11 17:16:00.719 UTC [3388] WARNING: there is no transaction in progress
2026-08-11 17:16:00.734 UTC [3388] WARNING: there is no transaction in progress
2026-08-11 17:16:00.744 UTC [3388] WARNING: there is no transaction in progress
2026-08-11 17:16:00.752 UTC [3388] WARNING: there is no transaction in progress
2026-08-11 17:16:00.759 UTC [3388] WARNING: there is no transaction in progress
2026-08-11 17:16:00.767 UTC [3388] WARNING: there is no transaction in progress
2026-08-11 17:16:01,536 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 17:16:01,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 17:16:01,774 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 17:16:01,774 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 17:16:02,042 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 17:16:02,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 17:16:02,306 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 17:16:02,306 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 17:16:02,569 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 17:16:02,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-fd7l.c.gcp-upstream.internal';"
2026-08-11 17:16:03,086 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 17:16:03,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 17:16:03,353 [I] DB_PATCHES have already been applied
I20260811 17:16:03.358835 21722 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ranger-admin:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task2dz75L/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 17:16:03.358911 21722 mini_ranger.cc:193] Using host: 127.21.54.148
I20260811 17:16:03.363680 21722 mini_ranger.cc:227] Ranger admin URL: http://127.21.54.148:34383
Aug 11, 2026 5:16:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:16:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:16:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 5:16:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 5:16:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 5:16:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 5:16:04 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 17:16:04.627920 21722 mini_ranger.cc:160] Time spent starting Ranger: real 9.269s user 0.000s sys 0.013s
Aug 11, 2026 5:16:25 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 5:16:25 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 5:16:25 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 5:16:25 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 5:16:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 5:16:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 5:16:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 5:16:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 5:16:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 5:16:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 5:16:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 5:16:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 5:16:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 5:16:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 5:16:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 5:16:25 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 5:16:25 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 5:16:26 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 17:16:27.530855 21722 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.54.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.54.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.54.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/rangerkms_127.21.54.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.54.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/rangerkms_127.21.54.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.54.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.54.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.54.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/HTTP_127.21.54.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.54.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/HTTP_127.21.54.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 17:16:27 dist-test-slave-fd7l krb5kdc[3142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468587, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 17:16:27.647661 21722 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 17:16:27.660823 21722 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 17:16:28.710637 21722 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 17:16:28.710729 21722 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ranger-kms
I20260811 17:16:28.710750 21722 mini_ranger_kms.cc:227] Using postgres at 127.21.54.148:51495
2026-08-11 17:16:28,892 [I] DB FLAVOR :POSTGRES
2026-08-11 17:16:28,893 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 17:16:28,893 [I] Checking connection
2026-08-11 17:16:28,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 17:16:29,190 [I] connection success
2026-08-11 17:16:29,190 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 17:16:29,191 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 17:16:29,191 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 17:16:29,511 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 17:16:29,512 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 17:16:29,512 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 17:16:29,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.54.148:51495/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 17:16:29,833 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 17:16:30.408764 21722 mini_ranger_kms.cc:310] Created kms service
I20260811 17:16:30.632344 21722 mini_ranger_kms.cc:326] Created kudu user
I20260811 17:16:30.681353 21722 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 17:16:30 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468587, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
I20260811 17:16:31.145800 21722 mini_ranger_kms.cc:384] Added ranger policy
I20260811 17:16:31.145988 21722 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ranger-kms:/tmp/dist-test-task2dz75L/build/debug/bin/postgresql.jar:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task2dz75L/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 17:16:31.146026 21722 mini_ranger_kms.cc:240] Using host: 127.21.54.148
I20260811 17:16:31.151381 21722 mini_ranger_kms.cc:276] Ranger KMS PID: 3594
I20260811 17:16:31.151502 21722 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.54.148:37287
Aug 11, 2026 5:16:32 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:16:32 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:16:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 5:16:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 5:16:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 5:16:32 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 17:16:33 dist-test-slave-fd7l krb5kdc[3142](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786468593, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.54.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:33.663147 21722 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.015s user 0.000s sys 0.167s
I20260811 17:16:33.663280 21722 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 17:16:33.663352 21722 mini_ranger_kms.cc:401] 127.21.54.148:37287/kms/v1/keys
Aug 11 17:16:33 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468593, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.54.148@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
Aug 11 17:16:34 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468593, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.54.148@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
Aug 11 17:16:36 dist-test-slave-fd7l krb5kdc[3142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:16:36.573346 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:42533
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:42533
--ranger_config_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.54.148:37287/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:16:36.703261 3652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:36.703536 3652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:36.703577 3652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:36.707845 3652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:16:36.707973 3652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:36.708024 3652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:36.708061 3652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:16:36.708098 3652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:16:36.713918 3652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.54.148:37287/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:42533
--ranger_config_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:42533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:36.715085 3652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:36.716203 3652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:16:36.726147 3652 server_base.cc:1034] running on GCE node
W20260811 17:16:36.726199 3660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:36.726199 3658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:36.726132 3662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:36.727176 3652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:36.728498 3652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:36.729774 3652 hybrid_clock.cc:648] HybridClock initialized: now 1786468596729746 us; error 41 us; skew 500 ppm
Aug 11 17:16:36 dist-test-slave-fd7l krb5kdc[3142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468596, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:36.739267 3652 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:16:36.740305 3652 webserver.cc:460] Webserver started at http://127.21.54.190:46433/ using document root <none> and password file <none>
I20260811 17:16:36.740967 3652 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:36.741046 3652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:36.741250 3652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 17:16:36 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468596, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
I20260811 17:16:36.808738 3652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "5594ba99667d43dba42a02d124f99e3c"
format_stamp: "Formatted at 2026-08-11 17:16:36 on dist-test-slave-fd7l"
server_key: "4265e58f60ed130eee77f66e17906146"
server_key_iv: "f0b5291081cf56b9ef11491532ebb79d"
server_key_version: "kuduclusterkey@0"
I20260811 17:16:36.809476 3652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "5594ba99667d43dba42a02d124f99e3c"
format_stamp: "Formatted at 2026-08-11 17:16:36 on dist-test-slave-fd7l"
server_key: "4265e58f60ed130eee77f66e17906146"
server_key_iv: "f0b5291081cf56b9ef11491532ebb79d"
server_key_version: "kuduclusterkey@0"
I20260811 17:16:36.813668 3652 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:16:36.829818 3670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:36.831331 3652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:16:36.831485 3652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "5594ba99667d43dba42a02d124f99e3c"
format_stamp: "Formatted at 2026-08-11 17:16:36 on dist-test-slave-fd7l"
server_key: "4265e58f60ed130eee77f66e17906146"
server_key_iv: "f0b5291081cf56b9ef11491532ebb79d"
server_key_version: "kuduclusterkey@0"
I20260811 17:16:36.831640 3652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:36.845366 3652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:36.848616 3652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:36.848832 3652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:36.857379 3652 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:42533
I20260811 17:16:36.857396 3734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:42533 every 8 connection(s)
I20260811 17:16:36.858466 3652 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:16:36.861495 3735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:16:36.863997 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 3652
I20260811 17:16:36.864154 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:16:36.864475 21722 external_mini_cluster.cc:1428] Setting key 684fcfa54ac73924c45ddc443dba4b6c
I20260811 17:16:36.868162 3735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c: Bootstrap starting.
I20260811 17:16:36.870905 3735 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c: Neither blocks nor log segments found. Creating new log.
I20260811 17:16:36.871943 3735 log.cc:824] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c: Log is configured to *not* fsync() on all Append() calls
I20260811 17:16:36.874534 3735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c: No bootstrap required, opened a new log
Aug 11 17:16:36 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:16:36.879433 3735 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5594ba99667d43dba42a02d124f99e3c" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42533 } }
I20260811 17:16:36.879765 3735 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:16:36.879846 3735 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5594ba99667d43dba42a02d124f99e3c, State: Initialized, Role: FOLLOWER
I20260811 17:16:36.880398 3735 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5594ba99667d43dba42a02d124f99e3c" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42533 } }
I20260811 17:16:36.880625 3735 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:16:36.880725 3735 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:16:36.880843 3735 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:16:36.882108 3735 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5594ba99667d43dba42a02d124f99e3c" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42533 } }
I20260811 17:16:36.882536 3735 leader_election.cc:302] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5594ba99667d43dba42a02d124f99e3c; no voters:
I20260811 17:16:36.882900 3735 leader_election.cc:288] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:16:36.882948 3738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:36.866071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57916 (local address 127.21.54.190:42533)
0811 17:16:36.866597 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:36.866619 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:36.866664 (+ 45us) server_negotiation.cc:407] Connection header received
0811 17:16:36.867541 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:36.867570 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:36.868024 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:36.868496 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:36.869478 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:36.870995 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:36.872323 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:36.873126 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:36.873302 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:36.876580 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:36.876587 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:36.876606 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:36.876639 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:36.878753 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:36.879315 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:36.879321 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:36.879327 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:36.879476 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:36.879819 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:36.879823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:36.879825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:36.880421 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:36.880691 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:36.881527 (+ 836us) server_negotiation.cc:299] Negotiation successful
0811 17:16:36.881840 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":158,"threads_started":1}
I20260811 17:16:36.883085 3740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:16:36.884359 3735 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:16:36.884765 3740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [term 1 LEADER]: Becoming Leader. State: Replica: 5594ba99667d43dba42a02d124f99e3c, State: Running, Role: LEADER
I20260811 17:16:36.885174 3740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5594ba99667d43dba42a02d124f99e3c" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42533 } }
I20260811 17:16:36.887650 3741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5594ba99667d43dba42a02d124f99e3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5594ba99667d43dba42a02d124f99e3c" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42533 } } }
I20260811 17:16:36.887799 3741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [sys.catalog]: This master's current role is: LEADER
I20260811 17:16:36.887720 3735 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 17:16:36.888257 3742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5594ba99667d43dba42a02d124f99e3c. Latest consensus state: current_term: 1 leader_uuid: "5594ba99667d43dba42a02d124f99e3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5594ba99667d43dba42a02d124f99e3c" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42533 } } }
I20260811 17:16:36.888360 3742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [sys.catalog]: This master's current role is: LEADER
I20260811 17:16:36.888794 3755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:16:36.891500 3755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:16:36.904156 3755 catalog_manager.cc:1347] Generated new cluster ID: 5ce759a89a0f492b8d059b27d73aad8a
I20260811 17:16:36.904363 3755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:16:36.916495 3755 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:16:36.917701 3755 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:16:36.927946 3755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c: Generated new TSK 0
I20260811 17:16:36.928884 3755 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
I20260811 17:16:37.800964 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.54.148:37287/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:42533
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:16:37.946421 3782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:37.946730 3782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:37.946775 3782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:37.952603 3782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:37.952700 3782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:37.952802 3782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:16:37.962405 3782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.54.148:37287/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:42533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:37.964336 3782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:37.965947 3782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:37.979425 3792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:37.979425 3789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:37.980041 3790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:37.980044 3782 server_base.cc:1034] running on GCE node
I20260811 17:16:37.980859 3782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:37.981778 3782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:37.982998 3782 hybrid_clock.cc:648] HybridClock initialized: now 1786468597982948 us; error 62 us; skew 500 ppm
Aug 11 17:16:37 dist-test-slave-fd7l krb5kdc[3142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468597, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:37.987566 3782 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:16:37.989789 3782 webserver.cc:460] Webserver started at http://127.21.54.129:32979/ using document root <none> and password file <none>
I20260811 17:16:37.990658 3782 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:37.990761 3782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:37.991079 3782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 17:16:37 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468597, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
I20260811 17:16:38.021792 3782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "0e181c84d510484db7d652cd82b5a561"
format_stamp: "Formatted at 2026-08-11 17:16:38 on dist-test-slave-fd7l"
server_key: "8222ece0fbf412f487e15ba682f0fc1f"
server_key_iv: "8f98c8323791f85c0c800dd6bfba1512"
server_key_version: "kuduclusterkey@0"
I20260811 17:16:38.022648 3782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "0e181c84d510484db7d652cd82b5a561"
format_stamp: "Formatted at 2026-08-11 17:16:38 on dist-test-slave-fd7l"
server_key: "8222ece0fbf412f487e15ba682f0fc1f"
server_key_iv: "8f98c8323791f85c0c800dd6bfba1512"
server_key_version: "kuduclusterkey@0"
I20260811 17:16:38.028231 3782 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 17:16:38.047940 3803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:38.054112 3782 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.000s
I20260811 17:16:38.054417 3782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "0e181c84d510484db7d652cd82b5a561"
format_stamp: "Formatted at 2026-08-11 17:16:38 on dist-test-slave-fd7l"
server_key: "8222ece0fbf412f487e15ba682f0fc1f"
server_key_iv: "8f98c8323791f85c0c800dd6bfba1512"
server_key_version: "kuduclusterkey@0"
I20260811 17:16:38.054690 3782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:38.078632 3782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:38.083220 3782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:38.083539 3782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:38.084441 3782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:16:38.085842 3782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:16:38.086308 3782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:38.086477 3782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:16:38.086580 3782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:38.101814 3782 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:39433
I20260811 17:16:38.103502 3782 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:16:38.103529 3932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:39433 every 8 connection(s)
I20260811 17:16:38.113386 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 3782
I20260811 17:16:38.113538 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:16:38.113865 21722 external_mini_cluster.cc:1428] Setting key a808c6cad1de38deadcb718ca8dad635
Aug 11 17:16:38 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468597, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:16:38.129557 3936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:38.106278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:40487 (local address 127.21.54.190:42533)
0811 17:16:38.107116 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:38.107121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:38.107708 (+ 587us) server_negotiation.cc:407] Connection header received
0811 17:16:38.109076 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:38.109081 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:38.109150 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:38.109266 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:38.111101 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:38.112059 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:38.119626 (+ 7567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:38.120076 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:38.120175 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:38.124045 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:38.124049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:38.124052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:38.124085 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:38.126365 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:38.127224 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:38.127231 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:38.127236 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:38.127346 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:38.127845 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:38.127852 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:38.127857 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:38.128171 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:38.128273 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:38.129192 (+ 919us) server_negotiation.cc:299] Negotiation successful
0811 17:16:38.129347 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":111,"threads_started":1}
I20260811 17:16:38.131793 3937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:38.106868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:42533 (local address 127.21.54.129:40487)
0811 17:16:38.107508 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:38.107576 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:38.108794 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:38.109644 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:38.109655 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:38.110112 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:38.110870 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:38.110893 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:38.112263 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:38.112267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:38.119400 (+ 7133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:38.119423 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:38.120489 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:38.120494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:38.120798 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:38.121631 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:38.121657 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:38.123772 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:38.126588 (+ 2816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:38.126598 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:38.126624 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:38.127083 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:38.127476 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:38.127483 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:38.127488 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:38.127717 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:38.128365 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:38.128376 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:38.128856 (+ 480us) client_negotiation.cc:769] Sending connection context
0811 17:16:38.130109 (+ 1253us) client_negotiation.cc:240] Negotiation successful
0811 17:16:38.130560 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":179,"threads_started":1}
I20260811 17:16:38.133816 3933 heartbeater.cc:347] Connected to a master server at 127.21.54.190:42533
I20260811 17:16:38.134106 3933 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:38.134812 3933 heartbeater.cc:511] Master 127.21.54.190:42533 requested a full tablet report, sending...
I20260811 17:16:38.137166 3691 ts_manager.cc:194] Registered new tserver with Master: 0e181c84d510484db7d652cd82b5a561 (127.21.54.129:39433)
I20260811 17:16:38.138957 3691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:40487
WARNING: no policy specified for kudu/127.21.54.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
I20260811 17:16:38.189154 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.54.148:37287/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:42533
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 17:16:38 dist-test-slave-fd7l krb5kdc[3142](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786468598, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 17:16:38.373150 3940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:38.374010 3940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:38.374152 3940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:38.380463 3940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:38.380622 3940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:38.380762 3940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:16:38.389151 3940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.54.148:37287/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:42533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:38.391044 3940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:38.392567 3940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:38.409094 3947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:38.409281 3946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:38.409485 3950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:38.409679 3940 server_base.cc:1034] running on GCE node
I20260811 17:16:38.411298 3940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:38.412796 3940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:38.416177 3940 hybrid_clock.cc:648] HybridClock initialized: now 1786468598416028 us; error 150 us; skew 500 ppm
Aug 11 17:16:38 dist-test-slave-fd7l krb5kdc[3142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468598, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:38.423686 3940 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:16:38.426296 3940 webserver.cc:460] Webserver started at http://127.21.54.130:43229/ using document root <none> and password file <none>
I20260811 17:16:38.427714 3940 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:38.427940 3940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:38.428457 3940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 17:16:38 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468598, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
I20260811 17:16:38.469126 3940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "a70af3d945ec42c4b0b8fb9359a8b2d4"
format_stamp: "Formatted at 2026-08-11 17:16:38 on dist-test-slave-fd7l"
server_key: "eb8abf5a83bdc7a54a26499c559d6653"
server_key_iv: "1fb07d4cdfd7eed7e3ff2710723b2e20"
server_key_version: "kuduclusterkey@0"
I20260811 17:16:38.471381 3940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "a70af3d945ec42c4b0b8fb9359a8b2d4"
format_stamp: "Formatted at 2026-08-11 17:16:38 on dist-test-slave-fd7l"
server_key: "eb8abf5a83bdc7a54a26499c559d6653"
server_key_iv: "1fb07d4cdfd7eed7e3ff2710723b2e20"
server_key_version: "kuduclusterkey@0"
I20260811 17:16:38.484269 3940 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.008s sys 0.000s
I20260811 17:16:38.538410 3959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:38.542165 3940 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.003s sys 0.000s
I20260811 17:16:38.542528 3940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "a70af3d945ec42c4b0b8fb9359a8b2d4"
format_stamp: "Formatted at 2026-08-11 17:16:38 on dist-test-slave-fd7l"
server_key: "eb8abf5a83bdc7a54a26499c559d6653"
server_key_iv: "1fb07d4cdfd7eed7e3ff2710723b2e20"
server_key_version: "kuduclusterkey@0"
I20260811 17:16:38.542840 3940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:38.601668 3940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:38.609330 3940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:38.609907 3940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:38.611375 3940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:16:38.613711 3940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:16:38.613842 3940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:38.613955 3940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:16:38.614024 3940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:38.647621 3940 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:43727
I20260811 17:16:38.650398 3940 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:16:38.653101 4088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:43727 every 8 connection(s)
I20260811 17:16:38.659385 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 3940
I20260811 17:16:38.659754 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:16:38.660401 21722 external_mini_cluster.cc:1428] Setting key c1a09570a997ed8f600c63b67fb74c79
Aug 11 17:16:38 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468598, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:16:38.691820 4091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:38.657146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:42499 (local address 127.21.54.190:42533)
0811 17:16:38.658674 (+ 1528us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:38.658684 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:38.661754 (+ 3070us) server_negotiation.cc:407] Connection header received
0811 17:16:38.664368 (+ 2614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:38.664373 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:38.664468 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:38.664666 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:38.668008 (+ 3342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:38.669632 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:38.673172 (+ 3540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:38.673885 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:38.674044 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:38.680420 (+ 6376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:38.680426 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:38.680429 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:38.680608 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:38.684941 (+ 4333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:38.686676 (+ 1735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:38.686689 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:38.686691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:38.686884 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:38.687987 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:38.687991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:38.687993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:38.688655 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:38.688951 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:38.690717 (+ 1766us) server_negotiation.cc:299] Negotiation successful
0811 17:16:38.691024 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1332,"thread_start_us":245,"threads_started":1}
I20260811 17:16:38.696149 4092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:38.658524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:42533 (local address 127.21.54.130:42499)
0811 17:16:38.661194 (+ 2670us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:38.661415 (+ 221us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:38.663619 (+ 2204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:38.665347 (+ 1728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:38.665360 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:38.666163 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:38.667592 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:38.667649 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:38.669932 (+ 2283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:38.669935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:38.672769 (+ 2834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:38.672791 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:38.674475 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:38.674481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:38.675008 (+ 527us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:38.676396 (+ 1388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:38.676592 (+ 196us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:38.679973 (+ 3381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:38.685252 (+ 5279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:38.685261 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:38.685301 (+ 40us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:38.686275 (+ 974us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:38.687196 (+ 921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:38.687207 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:38.687209 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:38.687761 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:38.689142 (+ 1381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:38.689160 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:38.690064 (+ 904us) client_negotiation.cc:769] Sending connection context
0811 17:16:38.692397 (+ 2333us) client_negotiation.cc:240] Negotiation successful
0811 17:16:38.693401 (+ 1004us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2233,"thread_start_us":285,"threads_started":1}
I20260811 17:16:38.699332 4089 heartbeater.cc:347] Connected to a master server at 127.21.54.190:42533
I20260811 17:16:38.699957 4089 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:38.701455 4089 heartbeater.cc:511] Master 127.21.54.190:42533 requested a full tablet report, sending...
I20260811 17:16:38.704073 3691 ts_manager.cc:194] Registered new tserver with Master: a70af3d945ec42c4b0b8fb9359a8b2d4 (127.21.54.130:43727)
I20260811 17:16:38.705801 3691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:42499
WARNING: no policy specified for kudu/127.21.54.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
I20260811 17:16:38.755399 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.54.148:37287/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:42533
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:16:39.119704 4095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:39.120419 4095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:39.120602 4095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:39.131249 4095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:39.131464 4095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:39.131675 4095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:16:39.142638 3933 heartbeater.cc:503] Master 127.21.54.190:42533 was elected leader, sending a full tablet report...
I20260811 17:16:39.145473 4095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.54.148:37287/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:42533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:39.148469 4095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:39.151015 4095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:39.177946 4102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:39.179003 4101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:39.180289 4105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:39.181931 4095 server_base.cc:1034] running on GCE node
I20260811 17:16:39.182861 4095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:39.184407 4095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:39.187839 4095 hybrid_clock.cc:648] HybridClock initialized: now 1786468599187711 us; error 143 us; skew 500 ppm
Aug 11 17:16:39 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468598, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
Aug 11 17:16:39 dist-test-slave-fd7l krb5kdc[3142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468599, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:39.196017 4095 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:16:39.198581 4095 webserver.cc:460] Webserver started at http://127.21.54.131:38369/ using document root <none> and password file <none>
I20260811 17:16:39.200110 4095 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:39.200367 4095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:39.200976 4095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 17:16:39 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468599, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
I20260811 17:16:39.249290 4095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "9f1d7de992cb4e9f847032f451ed6025"
format_stamp: "Formatted at 2026-08-11 17:16:39 on dist-test-slave-fd7l"
server_key: "c6063e8f86e451b2b5d0503641c280d4"
server_key_iv: "adc85f1c95f7d3ca067b7978fca77d01"
server_key_version: "kuduclusterkey@0"
I20260811 17:16:39.251029 4095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "9f1d7de992cb4e9f847032f451ed6025"
format_stamp: "Formatted at 2026-08-11 17:16:39 on dist-test-slave-fd7l"
server_key: "c6063e8f86e451b2b5d0503641c280d4"
server_key_iv: "adc85f1c95f7d3ca067b7978fca77d01"
server_key_version: "kuduclusterkey@0"
I20260811 17:16:39.261992 4095 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.008s sys 0.000s
I20260811 17:16:39.305151 4112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:39.308604 4095 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.001s
I20260811 17:16:39.308868 4095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "9f1d7de992cb4e9f847032f451ed6025"
format_stamp: "Formatted at 2026-08-11 17:16:39 on dist-test-slave-fd7l"
server_key: "c6063e8f86e451b2b5d0503641c280d4"
server_key_iv: "adc85f1c95f7d3ca067b7978fca77d01"
server_key_version: "kuduclusterkey@0"
I20260811 17:16:39.309136 4095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:39.343295 4095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:39.352618 4095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:39.353170 4095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:39.355041 4095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:16:39.357645 4095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:16:39.357832 4095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:39.358003 4095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:16:39.358095 4095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:39.393357 4095 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:37871
I20260811 17:16:39.393718 4244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:37871 every 8 connection(s)
I20260811 17:16:39.395785 4095 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:16:39.396615 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 4095
I20260811 17:16:39.396874 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:16:39.397513 21722 external_mini_cluster.cc:1428] Setting key ec2c14a5acce7b989ffa7a1c6be8aafe
Aug 11 17:16:39 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468599, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:16:39.434520 4247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:39.400770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:34529 (local address 127.21.54.190:42533)
0811 17:16:39.402239 (+ 1469us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:39.402253 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:39.404595 (+ 2342us) server_negotiation.cc:407] Connection header received
0811 17:16:39.407309 (+ 2714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:39.407314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:39.407468 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:39.407704 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:39.411194 (+ 3490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:39.413128 (+ 1934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:39.416257 (+ 3129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:39.417089 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:39.417264 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:39.422668 (+ 5404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:39.422684 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:39.422698 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:39.422773 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:39.429644 (+ 6871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:39.431029 (+ 1385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:39.431033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:39.431036 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:39.431251 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:39.432055 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:39.432058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:39.432068 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:39.432501 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:39.432690 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:39.433954 (+ 1264us) server_negotiation.cc:299] Negotiation successful
0811 17:16:39.434206 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1291,"thread_start_us":174,"threads_started":1}
I20260811 17:16:39.436720 4248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:39.401970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:42533 (local address 127.21.54.131:34529)
0811 17:16:39.404237 (+ 2267us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:39.404394 (+ 157us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:39.406816 (+ 2422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:39.408121 (+ 1305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:39.408133 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:39.409293 (+ 1160us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:39.410875 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:39.410927 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:39.413387 (+ 2460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:39.413391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:39.415932 (+ 2541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:39.415954 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:39.417459 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:39.417465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:39.417855 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:39.419011 (+ 1156us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:39.419059 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:39.422215 (+ 3156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:39.429926 (+ 7711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:39.429945 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:39.429988 (+ 43us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:39.430804 (+ 816us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:39.431453 (+ 649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:39.431458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:39.431460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:39.431848 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:39.432729 (+ 881us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:39.432738 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:39.433389 (+ 651us) client_negotiation.cc:769] Sending connection context
0811 17:16:39.433797 (+ 408us) client_negotiation.cc:240] Negotiation successful
0811 17:16:39.434485 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1840,"thread_start_us":332,"threads_started":1}
I20260811 17:16:39.438932 4245 heartbeater.cc:347] Connected to a master server at 127.21.54.190:42533
I20260811 17:16:39.439446 4245 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:39.440614 4245 heartbeater.cc:511] Master 127.21.54.190:42533 requested a full tablet report, sending...
I20260811 17:16:39.442744 3691 ts_manager.cc:194] Registered new tserver with Master: 9f1d7de992cb4e9f847032f451ed6025 (127.21.54.131:37871)
I20260811 17:16:39.443760 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:16:39.443878 3691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:34529
I20260811 17:16:39.445142 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 3782
I20260811 17:16:39.484392 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 3940
I20260811 17:16:39.505888 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 4095
I20260811 17:16:39.538491 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 3652
I20260811 17:16:39.553369 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:42533
--webserver_interface=127.21.54.190
--webserver_port=46433
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:42533
--ranger_config_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.54.148:37287/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.54.148:37287/kms with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:16:39.711251 4249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:39.711537 4249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:39.711635 4249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:39.715853 4249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:16:39.715921 4249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:39.715941 4249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:39.715957 4249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:16:39.715972 4249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:16:39.722491 4249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.54.148:37287/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:42533
--ranger_config_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:42533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=46433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:39.723735 4249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:39.724906 4249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:39.733878 4255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:39.733911 4254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:39.733994 4249 server_base.cc:1034] running on GCE node
W20260811 17:16:39.733994 4258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:39.734803 4249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:39.736043 4249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:39.737236 4249 hybrid_clock.cc:648] HybridClock initialized: now 1786468599737220 us; error 29 us; skew 500 ppm
Aug 11 17:16:39 dist-test-slave-fd7l krb5kdc[3142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468599, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:39.740113 4249 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:16:39.741109 4249 webserver.cc:460] Webserver started at http://127.21.54.190:46433/ using document root <none> and password file <none>
I20260811 17:16:39.741676 4249 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:39.741778 4249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 17:16:39 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468599, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
I20260811 17:16:39.930333 4249 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:16:39.932785 4267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:39.934206 4249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:16:39.934360 4249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "5594ba99667d43dba42a02d124f99e3c"
format_stamp: "Formatted at 2026-08-11 17:16:36 on dist-test-slave-fd7l"
server_key: "4265e58f60ed130eee77f66e17906146"
server_key_iv: "f0b5291081cf56b9ef11491532ebb79d"
server_key_version: "kuduclusterkey@0"
I20260811 17:16:39.934952 4249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:39.960623 4249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:39.963938 4249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:39.964164 4249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:39.972241 4249 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:42533
I20260811 17:16:39.972251 4329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:42533 every 8 connection(s)
I20260811 17:16:39.973402 4249 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:16:39.977952 4330 sys_catalog.cc:263] Verifying existing consensus state
I20260811 17:16:39.978174 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 4249
I20260811 17:16:39.978709 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.54.148:37287/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:39433
--local_ip_for_outbound_sockets=127.21.54.129
--tserver_master_addrs=127.21.54.190:42533
--webserver_port=32979
--webserver_interface=127.21.54.129
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.54.148:37287/kms with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:16:39.980350 4330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c: Bootstrap starting.
I20260811 17:16:39.997939 4330 log.cc:824] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c: Log is configured to *not* fsync() on all Append() calls
I20260811 17:16:40.005090 4330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:16:40.005520 4330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c: Bootstrap complete.
I20260811 17:16:40.010057 4330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5594ba99667d43dba42a02d124f99e3c" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42533 } }
I20260811 17:16:40.010497 4330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5594ba99667d43dba42a02d124f99e3c, State: Initialized, Role: FOLLOWER
I20260811 17:16:40.011005 4330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5594ba99667d43dba42a02d124f99e3c" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42533 } }
I20260811 17:16:40.011171 4330 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:16:40.011257 4330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:16:40.011381 4330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [term 1 FOLLOWER]: Advancing to term 2
I20260811 17:16:40.013489 4330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5594ba99667d43dba42a02d124f99e3c" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42533 } }
I20260811 17:16:40.014106 4330 leader_election.cc:302] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5594ba99667d43dba42a02d124f99e3c; no voters:
I20260811 17:16:40.014514 4330 leader_election.cc:288] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 17:16:40.014636 4336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [term 2 FOLLOWER]: Leader election won for term 2
I20260811 17:16:40.014823 4336 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [term 2 LEADER]: Becoming Leader. State: Replica: 5594ba99667d43dba42a02d124f99e3c, State: Running, Role: LEADER
I20260811 17:16:40.015134 4336 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5594ba99667d43dba42a02d124f99e3c" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42533 } }
I20260811 17:16:40.015828 4330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:16:40.016561 4337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5594ba99667d43dba42a02d124f99e3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5594ba99667d43dba42a02d124f99e3c" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42533 } } }
I20260811 17:16:40.016714 4337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [sys.catalog]: This master's current role is: LEADER
I20260811 17:16:40.017007 4338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5594ba99667d43dba42a02d124f99e3c. Latest consensus state: current_term: 2 leader_uuid: "5594ba99667d43dba42a02d124f99e3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5594ba99667d43dba42a02d124f99e3c" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42533 } } }
I20260811 17:16:40.017091 4338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c [sys.catalog]: This master's current role is: LEADER
I20260811 17:16:40.017551 4343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:16:40.020741 4343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:16:40.021379 4343 catalog_manager.cc:1259] Loaded cluster ID: 5ce759a89a0f492b8d059b27d73aad8a
I20260811 17:16:40.021454 4343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:16:40.021407 4330 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 17:16:40.023850 4343 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:16:40.024580 4343 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c: Loaded TSK: 0
I20260811 17:16:40.025836 4343 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 17:16:40.131884 4333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:40.132261 4333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:40.132329 4333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:40.138803 4333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:40.138958 4333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:40.139089 4333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:16:40.146250 4333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.54.148:37287/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:39433
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=32979
--tserver_master_addrs=127.21.54.190:42533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:40.147775 4333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:40.149297 4333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:40.162497 4377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:40.162524 4374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:40.163058 4375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:40.164023 4333 server_base.cc:1034] running on GCE node
I20260811 17:16:40.164664 4333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:40.165522 4333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:40.168836 4333 hybrid_clock.cc:648] HybridClock initialized: now 1786468600168576 us; error 271 us; skew 500 ppm
Aug 11 17:16:40 dist-test-slave-fd7l krb5kdc[3142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:40.172622 4333 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:16:40.173951 4333 webserver.cc:460] Webserver started at http://127.21.54.129:32979/ using document root <none> and password file <none>
I20260811 17:16:40.174793 4333 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:40.174964 4333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 17:16:40 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
I20260811 17:16:40.195659 4333 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:16:40.199402 4386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:40.201238 4333 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 17:16:40.201383 4333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "0e181c84d510484db7d652cd82b5a561"
format_stamp: "Formatted at 2026-08-11 17:16:38 on dist-test-slave-fd7l"
server_key: "8222ece0fbf412f487e15ba682f0fc1f"
server_key_iv: "8f98c8323791f85c0c800dd6bfba1512"
server_key_version: "kuduclusterkey@0"
I20260811 17:16:40.202209 4333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:40.219899 4333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:40.223892 4333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:40.224181 4333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:40.225059 4333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:16:40.226038 4333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:16:40.226090 4333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:40.226133 4333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:16:40.226190 4333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:40.237514 4333 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:39433
I20260811 17:16:40.237609 4517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:39433 every 8 connection(s)
I20260811 17:16:40.238951 4333 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:16:40.239975 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 4333
I20260811 17:16:40.240655 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.54.148:37287/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:43727
--local_ip_for_outbound_sockets=127.21.54.130
--tserver_master_addrs=127.21.54.190:42533
--webserver_port=43229
--webserver_interface=127.21.54.130
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.54.148:37287/kms with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 17:16:40 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:16:40.273859 4525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:40.245703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:58943 (local address 127.21.54.190:42533)
0811 17:16:40.249178 (+ 3475us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:40.249202 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:40.251175 (+ 1973us) server_negotiation.cc:407] Connection header received
0811 17:16:40.251512 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:40.251527 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:40.252008 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:40.252385 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:40.254490 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:40.256077 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:40.257867 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:40.258597 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:40.258769 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:40.266988 (+ 8219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:40.266997 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:40.267020 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:40.267063 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:40.269609 (+ 2546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:40.270425 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:40.270435 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:40.270441 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:40.270614 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:40.271096 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:40.271100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:40.271102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:40.271673 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:40.271818 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:40.272631 (+ 813us) server_negotiation.cc:299] Negotiation successful
0811 17:16:40.272933 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3263,"thread_start_us":131,"threads_started":1}
I20260811 17:16:40.273870 4522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:40.244026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:42533 (local address 127.21.54.129:58943)
0811 17:16:40.249549 (+ 5523us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:40.249632 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:40.250800 (+ 1168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:40.253164 (+ 2364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:40.253173 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:40.253583 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:40.254267 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:40.254291 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:40.256253 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:40.256257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:40.257712 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:40.257725 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:40.259012 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:40.259016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:40.259218 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:40.264731 (+ 5513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:40.264757 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:40.266649 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:40.269799 (+ 3150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:40.269806 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:40.269823 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:40.270252 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:40.270739 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:40.270743 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:40.270745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:40.270969 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:40.271822 (+ 853us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:40.271830 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:40.272174 (+ 344us) client_negotiation.cc:769] Sending connection context
0811 17:16:40.272398 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 17:16:40.272813 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5251,"thread_start_us":160,"threads_started":1}
I20260811 17:16:40.277019 4518 heartbeater.cc:347] Connected to a master server at 127.21.54.190:42533
I20260811 17:16:40.277287 4518 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:40.277859 4518 heartbeater.cc:511] Master 127.21.54.190:42533 requested a full tablet report, sending...
W20260811 17:16:40.280289 4518 heartbeater.cc:644] Failed to heartbeat to 127.21.54.190:42533 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 17:16:40.280418 4518 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:40.280625 4518 heartbeater.cc:511] Master 127.21.54.190:42533 requested a full tablet report, sending...
I20260811 17:16:40.281610 4518 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:40.281780 4518 heartbeater.cc:511] Master 127.21.54.190:42533 requested a full tablet report, sending...
W20260811 17:16:40.282737 4518 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 17:16:40.402732 4521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:40.403198 4521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:40.403307 4521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:40.407843 4521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:40.407956 4521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:40.408058 4521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:16:40.414563 4521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.54.148:37287/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:43727
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=43229
--tserver_master_addrs=127.21.54.190:42533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:40.416018 4521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:40.417541 4521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:40.428897 4533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:40.428877 4532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:40.429729 4535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:40.429739 4521 server_base.cc:1034] running on GCE node
I20260811 17:16:40.430337 4521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:40.431212 4521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:40.432394 4521 hybrid_clock.cc:648] HybridClock initialized: now 1786468600432367 us; error 54 us; skew 500 ppm
Aug 11 17:16:40 dist-test-slave-fd7l krb5kdc[3142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:40.435258 4521 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:16:40.436435 4521 webserver.cc:460] Webserver started at http://127.21.54.130:43229/ using document root <none> and password file <none>
I20260811 17:16:40.437227 4521 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:40.437345 4521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 17:16:40 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
I20260811 17:16:40.457472 4521 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260811 17:16:40.460353 4544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:40.462037 4521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:16:40.462183 4521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "a70af3d945ec42c4b0b8fb9359a8b2d4"
format_stamp: "Formatted at 2026-08-11 17:16:38 on dist-test-slave-fd7l"
server_key: "eb8abf5a83bdc7a54a26499c559d6653"
server_key_iv: "1fb07d4cdfd7eed7e3ff2710723b2e20"
server_key_version: "kuduclusterkey@0"
I20260811 17:16:40.462917 4521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:40.480940 4521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:40.484819 4521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:40.485046 4521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:40.485752 4521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:16:40.486860 4521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:16:40.486963 4521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:40.487051 4521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:16:40.487108 4521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:40.501340 4521 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:43727
I20260811 17:16:40.502780 4521 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:16:40.503197 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 4521
I20260811 17:16:40.503748 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.54.148:37287/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:37871
--local_ip_for_outbound_sockets=127.21.54.131
--tserver_master_addrs=127.21.54.190:42533
--webserver_port=38369
--webserver_interface=127.21.54.131
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.54.148:37287/kms with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:16:40.506688 4673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:43727 every 8 connection(s)
Aug 11 17:16:40 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:16:40.527040 4525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:40.511634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:45369 (local address 127.21.54.190:42533)
0811 17:16:40.511779 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:40.511784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:40.512340 (+ 556us) server_negotiation.cc:407] Connection header received
0811 17:16:40.513686 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:40.513692 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:40.513771 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:40.513882 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:40.515754 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:40.516709 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:40.518775 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:40.519206 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:40.519307 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:40.522666 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:40.522670 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:40.522673 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:40.522704 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:40.524470 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:40.525153 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:40.525157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:40.525159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:40.525252 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:40.525664 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:40.525668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:40.525670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:40.525902 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:40.525975 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:40.526714 (+ 739us) server_negotiation.cc:299] Negotiation successful
0811 17:16:40.526864 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:16:40.529088 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:40.511092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:42533 (local address 127.21.54.130:45369)
0811 17:16:40.512112 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:40.512174 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:40.513376 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:40.514267 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:40.514279 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:40.514718 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:40.515518 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:40.515544 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:40.516911 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:40.516917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:40.518588 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:40.518604 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:40.519569 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:40.519576 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:40.519854 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:40.520648 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:40.520671 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:40.522434 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:40.524644 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:40.524651 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:40.524670 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:40.525029 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:40.525364 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:40.525366 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:40.525368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:40.525544 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:40.526058 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:40.526066 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:40.526407 (+ 341us) client_negotiation.cc:769] Sending connection context
0811 17:16:40.527524 (+ 1117us) client_negotiation.cc:240] Negotiation successful
0811 17:16:40.527910 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"thread_start_us":167,"threads_started":1}
I20260811 17:16:40.530836 4675 heartbeater.cc:347] Connected to a master server at 127.21.54.190:42533
I20260811 17:16:40.531093 4675 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:40.531724 4675 heartbeater.cc:511] Master 127.21.54.190:42533 requested a full tablet report, sending...
W20260811 17:16:40.533658 4675 heartbeater.cc:644] Failed to heartbeat to 127.21.54.190:42533 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 17:16:40.533789 4675 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:40.533972 4675 heartbeater.cc:511] Master 127.21.54.190:42533 requested a full tablet report, sending...
I20260811 17:16:40.534883 4675 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:40.535049 4675 heartbeater.cc:511] Master 127.21.54.190:42533 requested a full tablet report, sending...
W20260811 17:16:40.535849 4675 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 17:16:40.665555 4674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:40.665896 4674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:40.665988 4674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:40.671262 4674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:40.671370 4674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:40.671494 4674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:16:40.678229 4674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.54.148:37287/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:37871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=38369
--tserver_master_addrs=127.21.54.190:42533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:40.679464 4674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:40.680626 4674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:40.693399 4690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:40.693423 4688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:40.693815 4687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:40.694490 4674 server_base.cc:1034] running on GCE node
I20260811 17:16:40.694981 4674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:40.695894 4674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:40.699265 4674 hybrid_clock.cc:648] HybridClock initialized: now 1786468600699171 us; error 106 us; skew 500 ppm
Aug 11 17:16:40 dist-test-slave-fd7l krb5kdc[3142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:40.703495 4674 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:16:40.704883 4674 webserver.cc:460] Webserver started at http://127.21.54.131:38369/ using document root <none> and password file <none>
I20260811 17:16:40.705740 4674 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:40.705875 4674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 17:16:40 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
I20260811 17:16:40.729969 4674 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.000s
I20260811 17:16:40.732831 4699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:40.734412 4674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:16:40.734586 4674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "9f1d7de992cb4e9f847032f451ed6025"
format_stamp: "Formatted at 2026-08-11 17:16:39 on dist-test-slave-fd7l"
server_key: "c6063e8f86e451b2b5d0503641c280d4"
server_key_iv: "adc85f1c95f7d3ca067b7978fca77d01"
server_key_version: "kuduclusterkey@0"
I20260811 17:16:40.735221 4674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:40.766405 4674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:40.770704 4674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:40.770949 4674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:40.771781 4674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:16:40.773041 4674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:16:40.773156 4674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 17:16:40.773304 4674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:16:40.773372 4674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:40.786216 4674 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:37871
I20260811 17:16:40.786293 4839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:37871 every 8 connection(s)
I20260811 17:16:40.787746 4674 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:16:40.791033 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 4674
Aug 11 17:16:40 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:16:40.808962 4844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:40.791798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57920 (local address 127.21.54.190:42533)
0811 17:16:40.792035 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:40.792047 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:40.792890 (+ 843us) server_negotiation.cc:407] Connection header received
0811 17:16:40.793138 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:40.793142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:40.793203 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:40.793274 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:40.794524 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:40.795425 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:40.796857 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:40.797241 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:40.797331 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:40.799499 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:40.799503 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:40.799506 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:40.799540 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:40.801895 (+ 2355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:40.806075 (+ 4180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:40.806082 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:40.806086 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:40.806200 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:40.806609 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:40.806614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:40.806616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:40.806898 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:40.807126 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:40.808138 (+ 1012us) server_negotiation.cc:299] Negotiation successful
0811 17:16:40.808296 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"spinlock_wait_cycles":2304,"thread_start_us":88,"threads_started":1}
I20260811 17:16:40.813669 4525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:40.789997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:48525 (local address 127.21.54.190:42533)
0811 17:16:40.790154 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:40.790159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:40.792370 (+ 2211us) server_negotiation.cc:407] Connection header received
0811 17:16:40.793783 (+ 1413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:40.793789 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:40.793845 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:40.793906 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:40.795745 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:40.796638 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:40.798610 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:40.799051 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:40.799145 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:40.808009 (+ 8864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:40.808016 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:40.808023 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:40.808099 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:40.810300 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:40.811414 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:40.811419 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:40.811422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:40.811518 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:40.812121 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:40.812126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:40.812127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:40.812386 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:40.812538 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:40.813336 (+ 798us) server_negotiation.cc:299] Negotiation successful
0811 17:16:40.813483 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63,"spinlock_wait_cycles":43008}
I20260811 17:16:40.815258 4843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:40.790559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:42533 (local address 127.21.54.131:48525)
0811 17:16:40.792086 (+ 1527us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:40.792146 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:40.793490 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:40.794410 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:40.794423 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:40.794828 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:40.795567 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:40.795591 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:40.796826 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:40.796833 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:40.798424 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:40.798440 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:40.799389 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:40.799397 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:40.799658 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:40.805193 (+ 5535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:40.805228 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:40.807797 (+ 2569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:40.810458 (+ 2661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:40.810466 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:40.810826 (+ 360us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:40.811281 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:40.811676 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:40.811683 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:40.811686 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:40.811970 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:40.812593 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:40.812602 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:40.813001 (+ 399us) client_negotiation.cc:769] Sending connection context
0811 17:16:40.813261 (+ 260us) client_negotiation.cc:240] Negotiation successful
0811 17:16:40.813675 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":938,"thread_start_us":248,"threads_started":1}
I20260811 17:16:40.816638 4841 heartbeater.cc:347] Connected to a master server at 127.21.54.190:42533
I20260811 17:16:40.816932 4841 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:40.817669 4841 heartbeater.cc:511] Master 127.21.54.190:42533 requested a full tablet report, sending...
W20260811 17:16:40.819491 4841 heartbeater.cc:644] Failed to heartbeat to 127.21.54.190:42533 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 17:16:40.819692 4841 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:40.819908 4841 heartbeater.cc:511] Master 127.21.54.190:42533 requested a full tablet report, sending...
I20260811 17:16:40.820827 4841 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:40.821014 4841 heartbeater.cc:511] Master 127.21.54.190:42533 requested a full tablet report, sending...
W20260811 17:16:40.821907 4841 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 17:16:41 dist-test-slave-fd7l krb5kdc[3142](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786468601, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:41.282976 4518 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:41.283294 4518 heartbeater.cc:511] Master 127.21.54.190:42533 requested a full tablet report, sending...
I20260811 17:16:41.284492 4286 ts_manager.cc:194] Registered new tserver with Master: 0e181c84d510484db7d652cd82b5a561 (127.21.54.129:39433)
I20260811 17:16:41.286231 4286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:58943
I20260811 17:16:41.536093 4675 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:41.536433 4675 heartbeater.cc:511] Master 127.21.54.190:42533 requested a full tablet report, sending...
I20260811 17:16:41.537389 4286 ts_manager.cc:194] Registered new tserver with Master: a70af3d945ec42c4b0b8fb9359a8b2d4 (127.21.54.130:43727)
I20260811 17:16:41.538292 4286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:45369
Aug 11 17:16:41 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468601, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
I20260811 17:16:41.822202 4841 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:41.822576 4841 heartbeater.cc:511] Master 127.21.54.190:42533 requested a full tablet report, sending...
I20260811 17:16:41.823599 4286 ts_manager.cc:194] Registered new tserver with Master: 9f1d7de992cb4e9f847032f451ed6025 (127.21.54.131:37871)
I20260811 17:16:41.824399 4286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:48525
I20260811 17:16:41.826088 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:16:41.846904 4865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:41.828123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57934 (local address 127.21.54.190:42533)
0811 17:16:41.828696 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:41.828701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:41.828716 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:16:41.828757 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:41.828759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:41.828807 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:41.828908 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:41.829893 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:41.830685 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:41.831875 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:41.832207 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:41.832306 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:41.840630 (+ 8324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:41.840634 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:41.840637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:41.840723 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:41.844015 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:41.844898 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:41.844902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:41.844904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:41.844999 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:41.845367 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:41.845371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:41.845373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:41.845625 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:41.845760 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:41.846560 (+ 800us) server_negotiation.cc:299] Negotiation successful
0811 17:16:41.846705 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":153,"threads_started":1}
I20260811 17:16:41.851403 4286 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:16:41.856251 4286 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:16:41.882870 4875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:41.873365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:43727 (local address 127.0.0.1:48174)
0811 17:16:41.873991 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:41.874009 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:41.874125 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:41.875659 (+ 1534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:41.875665 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:41.875691 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:16:41.875835 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:41.875842 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:41.877851 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:41.877855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:41.879390 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:41.879401 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:41.882110 (+ 2709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:41.882115 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:41.882262 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:41.882344 (+ 82us) client_negotiation.cc:769] Sending connection context
0811 17:16:41.882495 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 17:16:41.882651 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":4,"thread_start_us":161,"threads_started":1}
I20260811 17:16:41.883649 4876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:41.873573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48174 (local address 127.21.54.130:43727)
0811 17:16:41.874897 (+ 1324us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:41.874907 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:41.874934 (+ 27us) server_negotiation.cc:407] Connection header received
0811 17:16:41.875023 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:41.875029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:41.875261 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:41.875450 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:16:41.876028 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:41.877346 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:41.879551 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:41.881739 (+ 2188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:41.881877 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:41.882116 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:41.883310 (+ 1194us) server_negotiation.cc:299] Negotiation successful
0811 17:16:41.883442 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":99,"threads_started":1}
I20260811 17:16:41.883890 4874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:41.872844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50844 (local address 127.21.54.131:37871)
0811 17:16:41.878480 (+ 5636us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:41.878488 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:41.878515 (+ 27us) server_negotiation.cc:407] Connection header received
0811 17:16:41.878595 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:41.878600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:41.878799 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:41.878966 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:16:41.879274 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:41.880612 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:41.882002 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:41.882778 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:41.883403 (+ 625us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:41.883571 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:41.883623 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 17:16:41.883735 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5513,"thread_start_us":134,"threads_started":1}
I20260811 17:16:41.883289 4872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:41.871840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:37871 (local address 127.0.0.1:50844)
0811 17:16:41.872641 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:41.872697 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:41.872863 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:41.878958 (+ 6095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:41.878961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:41.878983 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:16:41.879083 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:41.879089 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:41.880752 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:41.880756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:41.881843 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:41.881855 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:41.882922 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:41.882926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:41.883017 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:41.883042 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 17:16:41.883091 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 17:16:41.883156 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":179,"threads_started":1}
I20260811 17:16:41.887590 4758 tablet_service.cc:1467] Processing CreateTablet for tablet 847606d70c7840a2b1f9460393cd93a7 (DEFAULT_TABLE table=test-table [id=53cca5d52c35443a91d86331c4701b7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:16:41.887588 4596 tablet_service.cc:1467] Processing CreateTablet for tablet 847606d70c7840a2b1f9460393cd93a7 (DEFAULT_TABLE table=test-table [id=53cca5d52c35443a91d86331c4701b7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:16:41.888820 4596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 847606d70c7840a2b1f9460393cd93a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:16:41.888849 4758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 847606d70c7840a2b1f9460393cd93a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:16:41.892261 4878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:41.874456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50850 (local address 127.21.54.129:39433)
0811 17:16:41.879320 (+ 4864us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:41.879327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:41.879352 (+ 25us) server_negotiation.cc:407] Connection header received
0811 17:16:41.879430 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:41.879437 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:41.879640 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:41.879785 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:16:41.885865 (+ 6080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:41.887405 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:41.889877 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:41.890892 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:41.891001 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:41.891205 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:41.891924 (+ 719us) server_negotiation.cc:299] Negotiation successful
0811 17:16:41.892065 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4753,"thread_start_us":201,"threads_started":1}
I20260811 17:16:41.893997 4877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:41.873822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:39433 (local address 127.0.0.1:50850)
0811 17:16:41.877601 (+ 3779us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:41.877618 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:41.877711 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:41.882199 (+ 4488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:41.882203 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:41.882224 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:16:41.885681 (+ 3457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:41.885692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:41.888287 (+ 2595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:41.888291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:41.889690 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:41.889705 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:41.891582 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:41.891586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:41.891693 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:41.891745 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 17:16:41.893721 (+ 1976us) client_negotiation.cc:240] Negotiation successful
0811 17:16:41.893828 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3693,"mutex_wait_us":3312,"thread_start_us":168,"threads_started":1}
I20260811 17:16:41.897843 4431 tablet_service.cc:1467] Processing CreateTablet for tablet 847606d70c7840a2b1f9460393cd93a7 (DEFAULT_TABLE table=test-table [id=53cca5d52c35443a91d86331c4701b7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:16:41.899152 4431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 847606d70c7840a2b1f9460393cd93a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:16:41.900496 4880 tablet_bootstrap.cc:492] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4: Bootstrap starting.
I20260811 17:16:41.903381 4880 tablet_bootstrap.cc:654] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4: Neither blocks nor log segments found. Creating new log.
I20260811 17:16:41.904409 4880 log.cc:824] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4: Log is configured to *not* fsync() on all Append() calls
I20260811 17:16:41.907285 4880 tablet_bootstrap.cc:492] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4: No bootstrap required, opened a new log
I20260811 17:16:41.907505 4880 ts_tablet_manager.cc:1397] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 17:16:41.910171 4881 tablet_bootstrap.cc:492] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025: Bootstrap starting.
I20260811 17:16:41.912422 4880 raft_consensus.cc:348] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d7de992cb4e9f847032f451ed6025" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37871 } } peers { permanent_uuid: "a70af3d945ec42c4b0b8fb9359a8b2d4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 43727 } } peers { permanent_uuid: "0e181c84d510484db7d652cd82b5a561" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39433 } }
I20260811 17:16:41.914166 4880 raft_consensus.cc:374] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:16:41.914283 4880 raft_consensus.cc:729] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a70af3d945ec42c4b0b8fb9359a8b2d4, State: Initialized, Role: FOLLOWER
I20260811 17:16:41.914885 4880 consensus_queue.cc:260] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d7de992cb4e9f847032f451ed6025" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37871 } } peers { permanent_uuid: "a70af3d945ec42c4b0b8fb9359a8b2d4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 43727 } } peers { permanent_uuid: "0e181c84d510484db7d652cd82b5a561" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39433 } }
I20260811 17:16:41.916363 4880 ts_tablet_manager.cc:1428] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260811 17:16:41.916853 4675 heartbeater.cc:503] Master 127.21.54.190:42533 was elected leader, sending a full tablet report...
I20260811 17:16:41.917812 4881 tablet_bootstrap.cc:654] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025: Neither blocks nor log segments found. Creating new log.
I20260811 17:16:41.919042 4881 log.cc:824] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025: Log is configured to *not* fsync() on all Append() calls
I20260811 17:16:41.920219 4883 tablet_bootstrap.cc:492] T 847606d70c7840a2b1f9460393cd93a7 P 0e181c84d510484db7d652cd82b5a561: Bootstrap starting.
I20260811 17:16:41.923457 4883 tablet_bootstrap.cc:654] T 847606d70c7840a2b1f9460393cd93a7 P 0e181c84d510484db7d652cd82b5a561: Neither blocks nor log segments found. Creating new log.
I20260811 17:16:41.924820 4883 log.cc:824] T 847606d70c7840a2b1f9460393cd93a7 P 0e181c84d510484db7d652cd82b5a561: Log is configured to *not* fsync() on all Append() calls
I20260811 17:16:41.927294 4881 tablet_bootstrap.cc:492] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025: No bootstrap required, opened a new log
I20260811 17:16:41.927572 4881 ts_tablet_manager.cc:1397] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.000s
I20260811 17:16:41.934500 4881 raft_consensus.cc:348] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d7de992cb4e9f847032f451ed6025" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37871 } } peers { permanent_uuid: "a70af3d945ec42c4b0b8fb9359a8b2d4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 43727 } } peers { permanent_uuid: "0e181c84d510484db7d652cd82b5a561" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39433 } }
I20260811 17:16:41.934971 4881 raft_consensus.cc:374] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:16:41.935082 4881 raft_consensus.cc:729] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1d7de992cb4e9f847032f451ed6025, State: Initialized, Role: FOLLOWER
I20260811 17:16:41.935660 4881 consensus_queue.cc:260] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d7de992cb4e9f847032f451ed6025" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37871 } } peers { permanent_uuid: "a70af3d945ec42c4b0b8fb9359a8b2d4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 43727 } } peers { permanent_uuid: "0e181c84d510484db7d652cd82b5a561" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39433 } }
I20260811 17:16:41.936291 4883 tablet_bootstrap.cc:492] T 847606d70c7840a2b1f9460393cd93a7 P 0e181c84d510484db7d652cd82b5a561: No bootstrap required, opened a new log
Aug 11 17:16:41 dist-test-slave-fd7l krb5kdc[3142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468601, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for HTTP/127.21.54.148@KRBTEST.COM
I20260811 17:16:41.937278 4881 ts_tablet_manager.cc:1428] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260811 17:16:41.937752 4841 heartbeater.cc:503] Master 127.21.54.190:42533 was elected leader, sending a full tablet report...
I20260811 17:16:41.938871 4883 ts_tablet_manager.cc:1397] T 847606d70c7840a2b1f9460393cd93a7 P 0e181c84d510484db7d652cd82b5a561: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.004s
I20260811 17:16:41.948227 4883 raft_consensus.cc:348] T 847606d70c7840a2b1f9460393cd93a7 P 0e181c84d510484db7d652cd82b5a561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d7de992cb4e9f847032f451ed6025" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37871 } } peers { permanent_uuid: "a70af3d945ec42c4b0b8fb9359a8b2d4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 43727 } } peers { permanent_uuid: "0e181c84d510484db7d652cd82b5a561" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39433 } }
I20260811 17:16:41.948719 4883 raft_consensus.cc:374] T 847606d70c7840a2b1f9460393cd93a7 P 0e181c84d510484db7d652cd82b5a561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:16:41.948840 4883 raft_consensus.cc:729] T 847606d70c7840a2b1f9460393cd93a7 P 0e181c84d510484db7d652cd82b5a561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e181c84d510484db7d652cd82b5a561, State: Initialized, Role: FOLLOWER
I20260811 17:16:41.949505 4883 consensus_queue.cc:260] T 847606d70c7840a2b1f9460393cd93a7 P 0e181c84d510484db7d652cd82b5a561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d7de992cb4e9f847032f451ed6025" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37871 } } peers { permanent_uuid: "a70af3d945ec42c4b0b8fb9359a8b2d4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 43727 } } peers { permanent_uuid: "0e181c84d510484db7d652cd82b5a561" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39433 } }
I20260811 17:16:41.951085 4883 ts_tablet_manager.cc:1428] T 847606d70c7840a2b1f9460393cd93a7 P 0e181c84d510484db7d652cd82b5a561: Time spent starting tablet: real 0.012s user 0.008s sys 0.000s
I20260811 17:16:41.951555 4518 heartbeater.cc:503] Master 127.21.54.190:42533 was elected leader, sending a full tablet report...
W20260811 17:16:41.992966 4520 tablet.cc:2369] T 847606d70c7840a2b1f9460393cd93a7 P 0e181c84d510484db7d652cd82b5a561: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:16:42.011668 4677 tablet.cc:2369] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:16:42.040040 4842 tablet.cc:2369] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:16:42.216450 4884 raft_consensus.cc:482] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:16:42.216779 4884 raft_consensus.cc:504] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d7de992cb4e9f847032f451ed6025" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37871 } } peers { permanent_uuid: "a70af3d945ec42c4b0b8fb9359a8b2d4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 43727 } } peers { permanent_uuid: "0e181c84d510484db7d652cd82b5a561" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39433 } }
I20260811 17:16:42.218369 4884 leader_election.cc:288] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1d7de992cb4e9f847032f451ed6025 (127.21.54.131:37871), 0e181c84d510484db7d652cd82b5a561 (127.21.54.129:39433)
I20260811 17:16:42.231113 4878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:42.220213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:42267 (local address 127.21.54.129:39433)
0811 17:16:42.220337 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:42.220341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:42.225992 (+ 5651us) server_negotiation.cc:407] Connection header received
0811 17:16:42.226215 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:42.226219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:42.226271 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:42.226335 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:16:42.226993 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:42.227796 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:42.229605 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:42.230221 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:42.230331 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:42.230389 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:42.230874 (+ 485us) server_negotiation.cc:299] Negotiation successful
0811 17:16:42.230961 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 17:16:42.231701 4894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:42.219455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:39433 (local address 127.21.54.130:42267)
0811 17:16:42.225885 (+ 6430us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:42.225902 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:42.226080 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:42.226487 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:42.226490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:42.226506 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:16:42.226840 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:42.226848 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:42.227958 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:42.227962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:42.229438 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:42.229450 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:42.230500 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:42.230504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:42.230711 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:42.230740 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 17:16:42.231306 (+ 566us) client_negotiation.cc:240] Negotiation successful
0811 17:16:42.231393 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6353,"thread_start_us":101,"threads_started":1}
I20260811 17:16:42.232656 4462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847606d70c7840a2b1f9460393cd93a7" candidate_uuid: "a70af3d945ec42c4b0b8fb9359a8b2d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e181c84d510484db7d652cd82b5a561" is_pre_election: true
I20260811 17:16:42.232872 4874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:42.220059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:42843 (local address 127.21.54.131:37871)
0811 17:16:42.220220 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:42.220225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:42.221877 (+ 1652us) server_negotiation.cc:407] Connection header received
0811 17:16:42.228701 (+ 6824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:42.228705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:42.228767 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:42.228823 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:16:42.229270 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:42.230033 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:42.231497 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:42.232107 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:42.232206 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:42.232255 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:42.232660 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 17:16:42.232747 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 17:16:42.233137 4462 raft_consensus.cc:2454] T 847606d70c7840a2b1f9460393cd93a7 P 0e181c84d510484db7d652cd82b5a561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a70af3d945ec42c4b0b8fb9359a8b2d4 in term 0.
I20260811 17:16:42.233247 4893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:42.218927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:37871 (local address 127.21.54.130:42843)
0811 17:16:42.221744 (+ 2817us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:42.221763 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:42.222284 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:42.228968 (+ 6684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:42.228971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:42.228989 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:16:42.229112 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:42.229120 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:42.230190 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:42.230193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:42.231341 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:42.231354 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:42.232357 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:42.232361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:42.232464 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:42.232487 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 17:16:42.233026 (+ 539us) client_negotiation.cc:240] Negotiation successful
0811 17:16:42.233106 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2678,"thread_start_us":212,"threads_started":1}
I20260811 17:16:42.233943 4547 leader_election.cc:302] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e181c84d510484db7d652cd82b5a561, a70af3d945ec42c4b0b8fb9359a8b2d4; no voters:
I20260811 17:16:42.234273 4884 raft_consensus.cc:2781] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:16:42.234375 4884 raft_consensus.cc:482] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:16:42.234422 4884 raft_consensus.cc:3037] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:16:42.235864 4884 raft_consensus.cc:504] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d7de992cb4e9f847032f451ed6025" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37871 } } peers { permanent_uuid: "a70af3d945ec42c4b0b8fb9359a8b2d4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 43727 } } peers { permanent_uuid: "0e181c84d510484db7d652cd82b5a561" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39433 } }
I20260811 17:16:42.236205 4785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847606d70c7840a2b1f9460393cd93a7" candidate_uuid: "a70af3d945ec42c4b0b8fb9359a8b2d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1d7de992cb4e9f847032f451ed6025" is_pre_election: true
I20260811 17:16:42.236313 4884 leader_election.cc:288] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1d7de992cb4e9f847032f451ed6025 (127.21.54.131:37871), 0e181c84d510484db7d652cd82b5a561 (127.21.54.129:39433)
I20260811 17:16:42.236580 4785 raft_consensus.cc:2454] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a70af3d945ec42c4b0b8fb9359a8b2d4 in term 0.
I20260811 17:16:42.237736 4462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847606d70c7840a2b1f9460393cd93a7" candidate_uuid: "a70af3d945ec42c4b0b8fb9359a8b2d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e181c84d510484db7d652cd82b5a561"
I20260811 17:16:42.237931 4462 raft_consensus.cc:3037] T 847606d70c7840a2b1f9460393cd93a7 P 0e181c84d510484db7d652cd82b5a561 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:16:42.237974 4785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847606d70c7840a2b1f9460393cd93a7" candidate_uuid: "a70af3d945ec42c4b0b8fb9359a8b2d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1d7de992cb4e9f847032f451ed6025"
I20260811 17:16:42.238101 4785 raft_consensus.cc:3037] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:16:42.239506 4785 raft_consensus.cc:2454] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a70af3d945ec42c4b0b8fb9359a8b2d4 in term 1.
I20260811 17:16:42.239506 4462 raft_consensus.cc:2454] T 847606d70c7840a2b1f9460393cd93a7 P 0e181c84d510484db7d652cd82b5a561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a70af3d945ec42c4b0b8fb9359a8b2d4 in term 1.
I20260811 17:16:42.239886 4547 leader_election.cc:302] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e181c84d510484db7d652cd82b5a561, a70af3d945ec42c4b0b8fb9359a8b2d4; no voters:
I20260811 17:16:42.240103 4884 raft_consensus.cc:2781] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:16:42.240412 4884 raft_consensus.cc:686] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [term 1 LEADER]: Becoming Leader. State: Replica: a70af3d945ec42c4b0b8fb9359a8b2d4, State: Running, Role: LEADER
I20260811 17:16:42.240962 4884 consensus_queue.cc:237] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d7de992cb4e9f847032f451ed6025" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37871 } } peers { permanent_uuid: "a70af3d945ec42c4b0b8fb9359a8b2d4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 43727 } } peers { permanent_uuid: "0e181c84d510484db7d652cd82b5a561" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39433 } }
I20260811 17:16:42.245100 4285 catalog_manager.cc:5495] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to a70af3d945ec42c4b0b8fb9359a8b2d4 (127.21.54.130). New cstate: current_term: 1 leader_uuid: "a70af3d945ec42c4b0b8fb9359a8b2d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1d7de992cb4e9f847032f451ed6025" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a70af3d945ec42c4b0b8fb9359a8b2d4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 43727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e181c84d510484db7d652cd82b5a561" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39433 } health_report { overall_health: UNKNOWN } } }
I20260811 17:16:42.345963 4876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:42.341989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48182 (local address 127.21.54.130:43727)
0811 17:16:42.342155 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:42.342159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:42.342173 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:16:42.342285 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:42.342287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:42.342343 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:42.342493 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:16:42.342687 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:42.343205 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:42.344178 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:42.344472 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:42.344633 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:42.344974 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:16:42.345434 (+ 460us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:16:42.345566 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:42.345647 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 17:16:42.345707 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 17:16:42.352941 4785 raft_consensus.cc:1261] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025 [term 1 FOLLOWER]: Refusing update from remote peer a70af3d945ec42c4b0b8fb9359a8b2d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:16:42.352980 4462 raft_consensus.cc:1261] T 847606d70c7840a2b1f9460393cd93a7 P 0e181c84d510484db7d652cd82b5a561 [term 1 FOLLOWER]: Refusing update from remote peer a70af3d945ec42c4b0b8fb9359a8b2d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:16:42.353776 4884 consensus_queue.cc:1035] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e181c84d510484db7d652cd82b5a561" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 39433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:16:42.354084 4898 consensus_queue.cc:1035] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1d7de992cb4e9f847032f451ed6025" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 37871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:16:42.361922 4909 mvcc.cc:205] Tried to move back new op lower bound from 7317375395229380608 to 7317375394782519296. Current Snapshot: MvccSnapshot[applied={T|T < 7317375395229380608}]
I20260811 17:16:42.363336 4912 mvcc.cc:205] Tried to move back new op lower bound from 7317375395229380608 to 7317375394782519296. Current Snapshot: MvccSnapshot[applied={T|T < 7317375395229380608}]
I20260811 17:16:42.365672 4911 mvcc.cc:205] Tried to move back new op lower bound from 7317375395229380608 to 7317375394782519296. Current Snapshot: MvccSnapshot[applied={T|T < 7317375395229380608 or (T in {7317375395229380608})}]
I20260811 17:16:42.371697 4285 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:16:42.371860 4285 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:16:42.374578 4285 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5594ba99667d43dba42a02d124f99e3c: Sending DeleteTablet for 3 replicas of tablet 847606d70c7840a2b1f9460393cd93a7
I20260811 17:16:42.375517 4596 tablet_service.cc:1514] Processing DeleteTablet for tablet 847606d70c7840a2b1f9460393cd93a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:16:42 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:48174
I20260811 17:16:42.375540 4758 tablet_service.cc:1514] Processing DeleteTablet for tablet 847606d70c7840a2b1f9460393cd93a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:16:42 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:50844
I20260811 17:16:42.375574 4431 tablet_service.cc:1514] Processing DeleteTablet for tablet 847606d70c7840a2b1f9460393cd93a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:16:42 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:50850
I20260811 17:16:42.376138 4920 tablet_replica.cc:330] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025: stopping tablet replica
I20260811 17:16:42.376510 4920 raft_consensus.cc:2229] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:16:42.376681 4919 tablet_replica.cc:330] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4: stopping tablet replica
I20260811 17:16:42.376930 4921 tablet_replica.cc:330] T 847606d70c7840a2b1f9460393cd93a7 P 0e181c84d510484db7d652cd82b5a561: stopping tablet replica
I20260811 17:16:42.377229 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 4333
I20260811 17:16:42.377336 4921 raft_consensus.cc:2229] T 847606d70c7840a2b1f9460393cd93a7 P 0e181c84d510484db7d652cd82b5a561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:16:42.377995 4920 raft_consensus.cc:2258] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:16:42.378443 4919 raft_consensus.cc:2229] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:16:42.378854 4919 raft_consensus.cc:2258] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:16:42.380823 4919 ts_tablet_manager.cc:1905] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:16:42.382571 4920 ts_tablet_manager.cc:1905] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:16:42.384642 4919 ts_tablet_manager.cc:1918] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:16:42.384795 4919 log.cc:1196] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/847606d70c7840a2b1f9460393cd93a7
I20260811 17:16:42.385202 4919 ts_tablet_manager.cc:1939] T 847606d70c7840a2b1f9460393cd93a7 P a70af3d945ec42c4b0b8fb9359a8b2d4: Deleting consensus metadata
I20260811 17:16:42.386059 4920 ts_tablet_manager.cc:1918] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:16:42.386237 4920 log.cc:1196] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/847606d70c7840a2b1f9460393cd93a7
I20260811 17:16:42.386636 4920 ts_tablet_manager.cc:1939] T 847606d70c7840a2b1f9460393cd93a7 P 9f1d7de992cb4e9f847032f451ed6025: Deleting consensus metadata
I20260811 17:16:42.386937 4271 catalog_manager.cc:4841] TS a70af3d945ec42c4b0b8fb9359a8b2d4 (127.21.54.130:43727): tablet 847606d70c7840a2b1f9460393cd93a7 (table test-table [id=53cca5d52c35443a91d86331c4701b7b]) successfully deleted
I20260811 17:16:42.387809 4271 catalog_manager.cc:4841] TS 9f1d7de992cb4e9f847032f451ed6025 (127.21.54.131:37871): tablet 847606d70c7840a2b1f9460393cd93a7 (table test-table [id=53cca5d52c35443a91d86331c4701b7b]) successfully deleted
W20260811 17:16:42.391026 4271 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.54.129:39433 (error 108)
I20260811 17:16:42.391177 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 4521
I20260811 17:16:42.395387 4877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:42.394985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:39433 (local address 127.0.0.1:50852)
0811 17:16:42.395188 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:42.395277 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.129:39433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260811 17:16:42.395618 4271 catalog_manager.cc:4584] TS 0e181c84d510484db7d652cd82b5a561 (127.21.54.129:39433): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 847606d70c7840a2b1f9460393cd93a7: Network error: Client connection negotiation failed: client connection to 127.21.54.129:39433: connect: Connection refused (error 111)
I20260811 17:16:42.403751 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 4674
I20260811 17:16:42.412451 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 4249
I20260811 17:16:42.423583 21722 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 17:16:42.769292 21722 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 17:16:42.769410 21722 mini_ranger.cc:66] Stopping Ranger...
I20260811 17:16:43.115345 21722 mini_ranger.cc:68] Stopped Ranger
2026-08-11 17:16:43.115 UTC [3175] LOG: received smart shutdown request
2026-08-11 17:16:43.120 UTC [3175] LOG: background worker "logical replication launcher" (PID 3184) exited with exit code 1
2026-08-11 17:16:43.120 UTC [3179] LOG: shutting down
2026-08-11 17:16:43.334 UTC [3175] LOG: database system is shut down
2026-08-11T17:16:43Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53403 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:16:43 dist-test-slave-fd7l krb5kdc[4931](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:16:43 dist-test-slave-fd7l krb5kdc[4931](info): set up 2 sockets
Aug 11 17:16:43 dist-test-slave-fd7l krb5kdc[4931](info): commencing operation
krb5kdc: starting...
W20260811 17:16:44.600796 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:16:44 dist-test-slave-fd7l krb5kdc[4931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:16:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:16:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:16:44.740634 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:46259
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:42541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:46259,127.21.54.189:40533,127.21.54.188:36963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:16:44.857213 4947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:44.857542 4947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:44.857630 4947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:44.861814 4947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:16:44.861907 4947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:44.861945 4947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:44.861987 4947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:16:44.862027 4947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:16:44.867444 4947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:42541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:46259,127.21.54.189:40533,127.21.54.188:36963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:46259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:44.868584 4947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:44.869630 4947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:44.878746 4955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:44.878757 4952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:44.878998 4947 server_base.cc:1034] running on GCE node
W20260811 17:16:44.878886 4953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:44.879757 4947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:44.881031 4947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:44.882232 4947 hybrid_clock.cc:648] HybridClock initialized: now 1786468604882211 us; error 31 us; skew 500 ppm
Aug 11 17:16:44 dist-test-slave-fd7l krb5kdc[4931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468604, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:44.885624 4947 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:16:44.886886 4947 webserver.cc:460] Webserver started at http://127.21.54.190:46785/ using document root <none> and password file <none>
I20260811 17:16:44.887737 4947 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:44.887835 4947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:44.888070 4947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:16:44.890334 4947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "bc764e78a99943f093f76dc58aeda774"
format_stamp: "Formatted at 2026-08-11 17:16:44 on dist-test-slave-fd7l"
server_key: "24a28d9abf3c5189863acac8a8fe24e0"
server_key_iv: "03c4e8770a1ffcfa132b6e1e4f6118dd"
server_key_version: "encryptionkey@0"
I20260811 17:16:44.891029 4947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "bc764e78a99943f093f76dc58aeda774"
format_stamp: "Formatted at 2026-08-11 17:16:44 on dist-test-slave-fd7l"
server_key: "24a28d9abf3c5189863acac8a8fe24e0"
server_key_iv: "03c4e8770a1ffcfa132b6e1e4f6118dd"
server_key_version: "encryptionkey@0"
I20260811 17:16:44.895617 4947 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 17:16:44.898370 4963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:44.899741 4947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:16:44.899914 4947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "bc764e78a99943f093f76dc58aeda774"
format_stamp: "Formatted at 2026-08-11 17:16:44 on dist-test-slave-fd7l"
server_key: "24a28d9abf3c5189863acac8a8fe24e0"
server_key_iv: "03c4e8770a1ffcfa132b6e1e4f6118dd"
server_key_version: "encryptionkey@0"
I20260811 17:16:44.900048 4947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:44.918318 4947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:44.921963 4947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:44.922205 4947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:44.929673 4947 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:46259
I20260811 17:16:44.929772 5026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:46259 every 8 connection(s)
I20260811 17:16:44.930685 4947 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:16:44.933529 5027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:16:44.937315 5027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:16:44.939222 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 4947
I20260811 17:16:44.939390 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:16:44.939713 21722 external_mini_cluster.cc:1428] Setting key 0e88a7b095167ba3ac10e0e282d40eca
Aug 11 17:16:44 dist-test-slave-fd7l krb5kdc[4931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468604, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.54.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.189@KRBTEST.COM" created.
I20260811 17:16:44.955190 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:44.939691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:46259 (local address 127.0.0.1:36622)
0811 17:16:44.940282 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:44.940340 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:44.941680 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:44.942595 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:44.942599 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:44.942936 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:44.943203 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:44.943216 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:44.944961 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:44.944964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:44.945897 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:44.945908 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:44.947009 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:44.947012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:44.947147 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:44.947846 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:44.947865 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:44.949560 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:44.951551 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:44.951560 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:44.951570 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:44.951943 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:44.952307 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:44.952310 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:44.952311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:44.952629 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:44.953585 (+ 956us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:44.953591 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:44.953722 (+ 131us) client_negotiation.cc:769] Sending connection context
0811 17:16:44.953875 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 17:16:44.954157 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":149,"threads_started":1}
I20260811 17:16:44.955188 5030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:44.939691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36622 (local address 127.21.54.190:46259)
0811 17:16:44.940267 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:44.940285 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:44.940466 (+ 181us) server_negotiation.cc:407] Connection header received
0811 17:16:44.942099 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:44.942108 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:44.942373 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:44.942583 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:44.943362 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:44.944793 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:44.946091 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:44.946781 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:44.946975 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:44.949723 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:44.949728 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:44.949738 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:44.949760 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:44.951443 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:44.952034 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:44.952039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:44.952040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:44.952179 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:44.952787 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:44.952789 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:44.952791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:44.953397 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:44.953575 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:44.954028 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 17:16:44.954231 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"spinlock_wait_cycles":53632,"thread_start_us":173,"threads_started":1}
I20260811 17:16:44.957832 5027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:16:44.958688 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:44.958401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:40533 (local address 127.0.0.1:37058)
0811 17:16:44.958530 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:44.958623 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:40533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 17:16:44.958894 4964 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.54.189:40533: connect: Connection refused (error 111)
I20260811 17:16:44.959851 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:44.959625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:40533 (local address 127.0.0.1:37070)
0811 17:16:44.959737 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:44.959786 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:40533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 17:16:44.960021 5027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.189:40533: Network error: Client connection negotiation failed: client connection to 127.21.54.189:40533: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.189.keytab.
Entry for principal kudu/127.21.54.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.189.keytab.
I20260811 17:16:44.972980 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.189:40533
--webserver_interface=127.21.54.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.189.keytab
--principal=kudu/127.21.54.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:42541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:46259,127.21.54.189:40533,127.21.54.188:36963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:16:45.009275 5027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } attempt: 1
I20260811 17:16:45.010249 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.009864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:40533 (local address 127.0.0.1:37082)
0811 17:16:45.010061 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.010141 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:40533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 17:16:45.011072 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.010815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:40533 (local address 127.0.0.1:37084)
0811 17:16:45.010936 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.010991 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:40533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 17:16:45.011265 5027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.189:40533: Network error: Client connection negotiation failed: client connection to 127.21.54.189:40533: connect: Connection refused (error 111)
I20260811 17:16:45.079465 5027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } attempt: 2
I20260811 17:16:45.080435 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.080060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:40533 (local address 127.0.0.1:37094)
0811 17:16:45.080262 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.080338 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:40533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 17:16:45.081286 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.081035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:40533 (local address 127.0.0.1:37096)
0811 17:16:45.081158 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.081213 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:40533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 17:16:45.081457 5027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.189:40533: Network error: Client connection negotiation failed: client connection to 127.21.54.189:40533: connect: Connection refused (error 111)
W20260811 17:16:45.091065 5034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:45.091475 5034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:45.091552 5034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:45.095710 5034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:16:45.095778 5034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:45.095798 5034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:45.095814 5034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:16:45.095829 5034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:16:45.101809 5034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:42541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:46259,127.21.54.189:40533,127.21.54.188:36963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.189.keytab
--principal=kudu/127.21.54.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.189:40533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.54.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:45.102876 5034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:45.103969 5034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:45.112915 5043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:45.112915 5040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:45.112941 5041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:45.113166 5034 server_base.cc:1034] running on GCE node
I20260811 17:16:45.113761 5034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:45.114862 5034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:45.116030 5034 hybrid_clock.cc:648] HybridClock initialized: now 1786468605116008 us; error 36 us; skew 500 ppm
Aug 11 17:16:45 dist-test-slave-fd7l krb5kdc[4931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468605, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:45.119095 5034 init.cc:377] Logged in from keytab as kudu/127.21.54.189@KRBTEST.COM (short username kudu)
I20260811 17:16:45.120198 5034 webserver.cc:460] Webserver started at http://127.21.54.189:45037/ using document root <none> and password file <none>
I20260811 17:16:45.120941 5034 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:45.121013 5034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:45.121251 5034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:16:45.123610 5034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-1/data/instance:
uuid: "f035bac3fc0e47a4ae2d049d16920e24"
format_stamp: "Formatted at 2026-08-11 17:16:45 on dist-test-slave-fd7l"
server_key: "1d0362dbb62cfd63eb20078fd85cca60"
server_key_iv: "9fdf59e451504bd2709eef1e205f3f4f"
server_key_version: "encryptionkey@0"
I20260811 17:16:45.124262 5034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-1/wal/instance:
uuid: "f035bac3fc0e47a4ae2d049d16920e24"
format_stamp: "Formatted at 2026-08-11 17:16:45 on dist-test-slave-fd7l"
server_key: "1d0362dbb62cfd63eb20078fd85cca60"
server_key_iv: "9fdf59e451504bd2709eef1e205f3f4f"
server_key_version: "encryptionkey@0"
I20260811 17:16:45.128402 5034 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 17:16:45.131214 5050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:45.132637 5034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:16:45.132778 5034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-1/wal
uuid: "f035bac3fc0e47a4ae2d049d16920e24"
format_stamp: "Formatted at 2026-08-11 17:16:45 on dist-test-slave-fd7l"
server_key: "1d0362dbb62cfd63eb20078fd85cca60"
server_key_iv: "9fdf59e451504bd2709eef1e205f3f4f"
server_key_version: "encryptionkey@0"
I20260811 17:16:45.132900 5034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:45.165103 5034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:45.169091 5034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:45.169332 5034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:45.172621 5027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } attempt: 3
I20260811 17:16:45.173537 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.173171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:40533 (local address 127.0.0.1:37112)
0811 17:16:45.173335 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.173437 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:40533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 17:16:45.174422 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.174186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:40533 (local address 127.0.0.1:37124)
0811 17:16:45.174308 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.174356 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:40533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 17:16:45.174641 5027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.189:40533: Network error: Client connection negotiation failed: client connection to 127.21.54.189:40533: connect: Connection refused (error 111)
I20260811 17:16:45.178112 5034 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.189:40533
I20260811 17:16:45.178139 5110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.189:40533 every 8 connection(s)
I20260811 17:16:45.179210 5034 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-1/data/info.pb
I20260811 17:16:45.181578 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 5034
I20260811 17:16:45.181742 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-1/wal/instance
I20260811 17:16:45.182081 21722 external_mini_cluster.cc:1428] Setting key 372948f19c06d749c10a2da5f276e04a
I20260811 17:16:45.182300 5111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:16:45.187449 5111 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.54.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.188@KRBTEST.COM" created.
Aug 11 17:16:45 dist-test-slave-fd7l krb5kdc[4931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468605, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.189@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:16:45.202698 5030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.189905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36628 (local address 127.21.54.190:46259)
0811 17:16:45.190052 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:45.190056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:45.191156 (+ 1100us) server_negotiation.cc:407] Connection header received
0811 17:16:45.192090 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:45.192094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:45.192161 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:45.192269 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:45.193546 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.194091 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.195260 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.195515 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.195635 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.198334 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:45.198337 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:45.198339 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:45.198366 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:45.200478 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.201077 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.201081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.201082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.201161 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.201511 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.201515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.201517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.201713 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:45.201826 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:45.202378 (+ 552us) server_negotiation.cc:299] Negotiation successful
0811 17:16:45.202516 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 17:16:45.203606 5115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.190465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:46259 (local address 127.0.0.1:36628)
0811 17:16:45.190992 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.191035 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:45.191868 (+ 833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:45.192439 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:45.192467 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:45.192905 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:45.193390 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:45.193404 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.194229 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.194232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:45.195125 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:45.195136 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.195679 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.195683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:45.195860 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.196502 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:45.196567 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:45.198136 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:45.200642 (+ 2506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:45.200649 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:45.200666 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:45.200943 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:45.201258 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:45.201261 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:45.201263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:45.201412 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:45.201826 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:45.201834 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:45.202105 (+ 271us) client_negotiation.cc:769] Sending connection context
0811 17:16:45.202319 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 17:16:45.202667 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":159,"threads_started":1}
I20260811 17:16:45.205400 5111 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:16:45 dist-test-slave-fd7l krb5kdc[4931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468605, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.189@KRBTEST.COM for kudu/127.21.54.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.188.keytab.
Entry for principal kudu/127.21.54.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.188.keytab.
I20260811 17:16:45.212572 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.188:36963
--webserver_interface=127.21.54.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.188.keytab
--principal=kudu/127.21.54.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:42541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:46259,127.21.54.189:40533,127.21.54.188:36963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:16:45.215827 5119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.206069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37128 (local address 127.21.54.189:40533)
0811 17:16:45.206369 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:45.206377 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:45.206393 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:16:45.206420 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:45.206424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:45.206609 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:45.206713 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:45.207181 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.208036 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.208904 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.209284 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.209423 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.211623 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:45.211632 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:45.211645 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:45.211667 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:45.213896 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.214337 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.214347 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.214350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.214463 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.214807 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.214809 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.214810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.215062 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:45.215180 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:45.215501 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 17:16:45.215679 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":225,"thread_start_us":155,"threads_started":1}
I20260811 17:16:45.215829 5115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.205919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:40533 (local address 127.0.0.1:37128)
0811 17:16:45.206082 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.206095 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:45.206161 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:45.206735 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:45.206738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:45.206874 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:45.207038 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:45.207048 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.208157 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.208160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:45.208786 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:45.208796 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.209448 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.209456 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:45.209528 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.209972 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:45.209988 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:45.211456 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:45.214016 (+ 2560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:45.214018 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:45.214020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:45.214217 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:45.214592 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:45.214598 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:45.214600 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:45.214709 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:45.215176 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:45.215180 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:45.215295 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 17:16:45.215418 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 17:16:45.215618 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 17:16:45.217953 5111 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:16:45.218770 5115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.218483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:36963 (local address 127.0.0.1:54496)
0811 17:16:45.218611 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.218692 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 17:16:45.218963 5051 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.54.188:36963: connect: Connection refused (error 111)
I20260811 17:16:45.219823 5115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.219602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:36963 (local address 127.0.0.1:54506)
0811 17:16:45.219714 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.219761 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 17:16:45.219993 5111 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:36963: Network error: Client connection negotiation failed: client connection to 127.21.54.188:36963: connect: Connection refused (error 111)
I20260811 17:16:45.269251 5111 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } attempt: 1
I20260811 17:16:45.270218 5115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.269882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:36963 (local address 127.0.0.1:54520)
0811 17:16:45.270052 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.270135 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 17:16:45.271013 5115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.270784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:36963 (local address 127.0.0.1:54532)
0811 17:16:45.270904 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.270957 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 17:16:45.271157 5111 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:36963: Network error: Client connection negotiation failed: client connection to 127.21.54.188:36963: connect: Connection refused (error 111)
I20260811 17:16:45.317821 5027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } attempt: 4
Aug 11 17:16:45 dist-test-slave-fd7l krb5kdc[4931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468604, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for kudu/127.21.54.189@KRBTEST.COM
I20260811 17:16:45.327497 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.318472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:40533 (local address 127.0.0.1:37136)
0811 17:16:45.318730 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.318748 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:45.318908 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:45.319216 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:45.319220 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:45.319434 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:45.319552 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:45.319558 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.320822 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.320827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:45.321386 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:45.321395 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.321898 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.321901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:45.322015 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.322583 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:45.322597 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:45.323899 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:45.325891 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:45.325894 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:45.325896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:45.326109 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:45.326408 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:45.326411 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:45.326413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:45.326504 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:45.326922 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:45.326925 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:45.327039 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 17:16:45.327154 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 17:16:45.327319 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260811 17:16:45.327497 5119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.318595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37136 (local address 127.21.54.189:40533)
0811 17:16:45.318730 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:45.318737 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:45.318837 (+ 100us) server_negotiation.cc:407] Connection header received
0811 17:16:45.319033 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:45.319036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:45.319102 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:45.319192 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:45.319692 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.320706 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.321514 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.321767 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.321878 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.324052 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:45.324056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:45.324058 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:45.324105 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:45.325761 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.326217 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.326221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.326223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.326307 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.326621 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.326625 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.326626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.326817 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:45.326930 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:45.327211 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 17:16:45.327329 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 17:16:45.328357 5027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:16:45.329113 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.328855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:36963 (local address 127.0.0.1:54540)
0811 17:16:45.328978 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.329032 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 17:16:45.329835 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.329573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:36963 (local address 127.0.0.1:54550)
0811 17:16:45.329724 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.329771 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 17:16:45.329957 5027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:36963: Network error: Client connection negotiation failed: client connection to 127.21.54.188:36963: connect: Connection refused (error 111)
W20260811 17:16:45.330720 5120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:45.330982 5120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:45.331024 5120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:45.335131 5120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:16:45.335189 5120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:45.335209 5120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:45.335225 5120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:16:45.335240 5120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:16:45.339316 5111 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } attempt: 2
I20260811 17:16:45.340119 5115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.339825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:36963 (local address 127.0.0.1:54562)
0811 17:16:45.339969 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.340034 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 17:16:45.340925 5115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.340708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:36963 (local address 127.0.0.1:54564)
0811 17:16:45.340818 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.340867 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 17:16:45.341107 5111 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:36963: Network error: Client connection negotiation failed: client connection to 127.21.54.188:36963: connect: Connection refused (error 111)
I20260811 17:16:45.341176 5120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:42541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:46259,127.21.54.189:40533,127.21.54.188:36963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.188.keytab
--principal=kudu/127.21.54.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.188:36963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.54.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:45.342271 5120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:45.343245 5120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:45.353250 5132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:45.353286 5120 server_base.cc:1034] running on GCE node
W20260811 17:16:45.353272 5128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:45.353250 5130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:45.354317 5120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:45.355535 5120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:45.356746 5120 hybrid_clock.cc:648] HybridClock initialized: now 1786468605356729 us; error 36 us; skew 500 ppm
Aug 11 17:16:45 dist-test-slave-fd7l krb5kdc[4931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468605, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:45.359658 5120 init.cc:377] Logged in from keytab as kudu/127.21.54.188@KRBTEST.COM (short username kudu)
I20260811 17:16:45.360625 5120 webserver.cc:460] Webserver started at http://127.21.54.188:37019/ using document root <none> and password file <none>
I20260811 17:16:45.361184 5120 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:45.361244 5120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:45.361420 5120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:16:45.363482 5120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-2/data/instance:
uuid: "34ac20fc757d4dd594e245e74d3d7da3"
format_stamp: "Formatted at 2026-08-11 17:16:45 on dist-test-slave-fd7l"
server_key: "c35f69dc05dbe80957f9f4fdb0645efa"
server_key_iv: "2443dd5356ce953a19fb71f81ee8e18a"
server_key_version: "encryptionkey@0"
I20260811 17:16:45.364022 5120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-2/wal/instance:
uuid: "34ac20fc757d4dd594e245e74d3d7da3"
format_stamp: "Formatted at 2026-08-11 17:16:45 on dist-test-slave-fd7l"
server_key: "c35f69dc05dbe80957f9f4fdb0645efa"
server_key_iv: "2443dd5356ce953a19fb71f81ee8e18a"
server_key_version: "encryptionkey@0"
I20260811 17:16:45.367772 5120 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:16:45.370271 5143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:45.371500 5120 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 17:16:45.371645 5120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-2/wal
uuid: "34ac20fc757d4dd594e245e74d3d7da3"
format_stamp: "Formatted at 2026-08-11 17:16:45 on dist-test-slave-fd7l"
server_key: "c35f69dc05dbe80957f9f4fdb0645efa"
server_key_iv: "2443dd5356ce953a19fb71f81ee8e18a"
server_key_version: "encryptionkey@0"
I20260811 17:16:45.371760 5120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:45.383023 5120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:45.386530 5120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:45.386776 5120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:45.389137 5027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } attempt: 1
I20260811 17:16:45.390091 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.389699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:36963 (local address 127.0.0.1:54568)
0811 17:16:45.389890 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.389971 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 17:16:45.391072 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.390751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:36963 (local address 127.0.0.1:54580)
0811 17:16:45.390888 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.390964 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:36963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 17:16:45.391395 5027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:36963: Network error: Client connection negotiation failed: client connection to 127.21.54.188:36963: connect: Connection refused (error 111)
I20260811 17:16:45.395279 5120 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.188:36963
I20260811 17:16:45.395310 5202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.188:36963 every 8 connection(s)
I20260811 17:16:45.396332 5120 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-2/data/info.pb
I20260811 17:16:45.399362 5203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:16:45.401338 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 5120
I20260811 17:16:45.401490 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468428054096-21722-0/minicluster-data/master-2/wal/instance
I20260811 17:16:45.401846 21722 external_mini_cluster.cc:1428] Setting key e97543f62ff1c2237dd3ded79a4e74d0
I20260811 17:16:45.404250 5203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:16:45 dist-test-slave-fd7l krb5kdc[4931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468605, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.188@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
Aug 11 17:16:45 dist-test-slave-fd7l krb5kdc[4931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.188@KRBTEST.COM
I20260811 17:16:45.416289 5030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.406577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36636 (local address 127.21.54.190:46259)
0811 17:16:45.406779 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:45.406784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:45.407035 (+ 251us) server_negotiation.cc:407] Connection header received
0811 17:16:45.407174 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:45.407177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:45.407229 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:45.407317 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:45.408051 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.408624 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.409582 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.409819 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.409931 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.412220 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:45.412224 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:45.412227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:45.412257 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:45.413887 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.414474 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.414476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.414478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.414551 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.414926 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.414932 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.414934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.415200 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:45.415276 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:45.415938 (+ 662us) server_negotiation.cc:299] Negotiation successful
0811 17:16:45.416076 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260811 17:16:45.417272 5208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.406499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:46259 (local address 127.0.0.1:36636)
0811 17:16:45.406897 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.406947 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:45.407070 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:45.407376 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:45.407380 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:45.407707 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:45.407885 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:45.407894 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.408747 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.408752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:45.409453 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:45.409463 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.409972 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.409976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:45.410088 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.410547 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:45.410563 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:45.412047 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:45.413996 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:45.414005 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:45.414018 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:45.414377 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:45.414647 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:45.414650 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:45.414651 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:45.414824 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:45.415304 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:45.415311 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:45.415649 (+ 338us) client_negotiation.cc:769] Sending connection context
0811 17:16:45.415862 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 17:16:45.416186 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":21,"thread_start_us":165,"threads_started":1}
I20260811 17:16:45.417275 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.403425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54590 (local address 127.21.54.188:36963)
0811 17:16:45.403948 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:45.403965 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:45.404004 (+ 39us) server_negotiation.cc:407] Connection header received
0811 17:16:45.404827 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:45.404847 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:45.405290 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:45.405678 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:45.406565 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.407899 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.408939 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.409420 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.409616 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.412709 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:45.412719 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:45.412735 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:45.412764 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:45.414674 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.415146 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.415150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.415152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.415220 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.415508 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.415511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.415513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.415776 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:45.415947 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:45.416200 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 17:16:45.416320 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":340,"thread_start_us":139,"threads_started":1}
I20260811 17:16:45.418646 5203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:16:45 dist-test-slave-fd7l krb5kdc[4931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468605, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.188@KRBTEST.COM for kudu/127.21.54.189@KRBTEST.COM
Aug 11 17:16:45 dist-test-slave-fd7l krb5kdc[4931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:16:45.428591 5208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.419145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:40533 (local address 127.0.0.1:37138)
0811 17:16:45.419294 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.419304 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:45.419611 (+ 307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:45.419985 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:45.419990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:45.420167 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:45.420289 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:45.420295 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.421128 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.421132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:45.421729 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:45.421738 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.422273 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.422277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:45.422380 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.422889 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:45.422900 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:45.424335 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:45.426442 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:45.426445 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:45.426448 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:45.426744 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:45.427142 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:45.427147 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:45.427149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:45.427255 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:45.427880 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:45.427884 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:45.428051 (+ 167us) client_negotiation.cc:769] Sending connection context
0811 17:16:45.428182 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 17:16:45.428341 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 17:16:45.428691 5119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.419287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37138 (local address 127.21.54.189:40533)
0811 17:16:45.419442 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:45.419446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:45.419462 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:16:45.419793 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:45.419797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:45.419863 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:45.419953 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:45.420410 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.421015 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.421862 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.422159 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.422240 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.424488 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:45.424491 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:45.424493 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:45.424574 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:45.426310 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.426916 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.426920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.426922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.427025 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.427365 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.427370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.427372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.427735 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:45.427893 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:45.428271 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 17:16:45.428382 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 17:16:45.429553 5203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:16:45.430047 5030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.420076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36646 (local address 127.21.54.190:46259)
0811 17:16:45.420195 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:45.420199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:45.420212 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:16:45.420286 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:45.420289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:45.420336 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:45.420412 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:45.421325 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.421834 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.422830 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.423045 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.423130 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.426084 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:45.426087 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:45.426089 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:45.426114 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:45.427938 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.428491 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.428495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.428497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.428642 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.429048 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.429055 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.429059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.429358 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:45.429621 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:45.429712 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 17:16:45.429844 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:16:45.432294 5111 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } attempt: 3
Aug 11 17:16:45 dist-test-slave-fd7l krb5kdc[4931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468605, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.188@KRBTEST.COM for kudu/127.21.54.188@KRBTEST.COM
Aug 11 17:16:45 dist-test-slave-fd7l krb5kdc[4931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.189@KRBTEST.COM
Aug 11 17:16:45 dist-test-slave-fd7l krb5kdc[4931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468605, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.189@KRBTEST.COM for kudu/127.21.54.188@KRBTEST.COM
I20260811 17:16:45.440826 5208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.430134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:36963 (local address 127.0.0.1:54604)
0811 17:16:45.430274 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.430292 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:45.430373 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:45.430948 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:45.430951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:45.431101 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:45.431247 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:45.431259 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.433031 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.433035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:45.433632 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:45.433641 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.434264 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.434267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:45.434349 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.434793 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:45.434809 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:45.436290 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:45.438595 (+ 2305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:45.438602 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:45.438606 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:45.438881 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:45.439620 (+ 739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:45.439626 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:45.439630 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:45.439761 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:45.440213 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:45.440217 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:45.440315 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 17:16:45.440436 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 17:16:45.440623 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 17:16:45.440826 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.430294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54604 (local address 127.21.54.188:36963)
0811 17:16:45.430735 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:45.430739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:45.430752 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:16:45.430781 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:45.430783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:45.430819 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:45.430927 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:45.431393 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.432392 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.433766 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.434135 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.434265 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.436447 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:45.436451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:45.436454 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:45.436477 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:45.438461 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.439359 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.439365 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.439369 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.439473 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.439868 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.439871 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.439872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.440081 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:45.440208 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:45.440496 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 17:16:45.440688 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":50}
I20260811 17:16:45.444032 5115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.432813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:36963 (local address 127.0.0.1:54606)
0811 17:16:45.432925 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.432943 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:45.433027 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:45.434149 (+ 1122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:45.434154 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:45.434365 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:45.434470 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:45.434477 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.435592 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.435597 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:45.436276 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:45.436287 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.437214 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.437221 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:45.437331 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.437934 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:45.437955 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:45.439656 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:45.442171 (+ 2515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:45.442175 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:45.442177 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:45.442422 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:45.442864 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:45.442870 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:45.442874 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:45.442972 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:45.443423 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:45.443427 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:45.443541 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 17:16:45.443661 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 17:16:45.443819 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 17:16:45.444051 5210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.432889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54606 (local address 127.21.54.188:36963)
0811 17:16:45.433909 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:45.433917 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:45.433927 (+ 10us) server_negotiation.cc:407] Connection header received
0811 17:16:45.433966 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:45.433968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:45.434010 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:45.434102 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:45.434577 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.435480 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.436411 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.436934 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.437132 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.439814 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:45.439822 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:45.439826 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:45.439856 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:45.442039 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.442678 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.442682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.442684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.442779 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.443085 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.443089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.443090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.443308 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:45.443432 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:45.443730 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 17:16:45.443856 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":949,"thread_start_us":170,"threads_started":1}
I20260811 17:16:45.444952 5119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.431540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37152 (local address 127.21.54.189:40533)
0811 17:16:45.431666 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:45.431670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:45.431685 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:16:45.431759 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:45.431763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:45.431809 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:45.431905 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:45.432674 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.433373 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.434270 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.434863 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.434928 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.439043 (+ 4115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:45.439049 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:45.439053 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:45.439084 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:45.441981 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.442585 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.442588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.442590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.442712 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.443085 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.443089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.443091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.443326 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:45.443594 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:45.444407 (+ 813us) server_negotiation.cc:299] Negotiation successful
0811 17:16:45.444587 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:16:45.445231 5203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3: Bootstrap starting.
I20260811 17:16:45.447371 5111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24: Bootstrap starting.
I20260811 17:16:45.448433 5203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3: Neither blocks nor log segments found. Creating new log.
I20260811 17:16:45.449321 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:16:45.449473 5203 log.cc:824] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3: Log is configured to *not* fsync() on all Append() calls
I20260811 17:16:45.450035 5111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24: Neither blocks nor log segments found. Creating new log.
I20260811 17:16:45.451030 5111 log.cc:824] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24: Log is configured to *not* fsync() on all Append() calls
I20260811 17:16:45.452039 5203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3: No bootstrap required, opened a new log
I20260811 17:16:45.453293 5111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24: No bootstrap required, opened a new log
I20260811 17:16:45.456996 5203 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } } peers { permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } } peers { permanent_uuid: "34ac20fc757d4dd594e245e74d3d7da3" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } }
I20260811 17:16:45.457446 5203 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:16:45.457576 5203 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34ac20fc757d4dd594e245e74d3d7da3, State: Initialized, Role: FOLLOWER
I20260811 17:16:45.457684 5111 raft_consensus.cc:348] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } } peers { permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } } peers { permanent_uuid: "34ac20fc757d4dd594e245e74d3d7da3" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } }
I20260811 17:16:45.457938 5111 raft_consensus.cc:374] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:16:45.458032 5111 raft_consensus.cc:729] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f035bac3fc0e47a4ae2d049d16920e24, State: Initialized, Role: FOLLOWER
I20260811 17:16:45.458586 5027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } attempt: 2
I20260811 17:16:45.458552 5111 consensus_queue.cc:260] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } } peers { permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } } peers { permanent_uuid: "34ac20fc757d4dd594e245e74d3d7da3" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } }
I20260811 17:16:45.458835 5203 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } } peers { permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } } peers { permanent_uuid: "34ac20fc757d4dd594e245e74d3d7da3" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } }
I20260811 17:16:45.460134 5111 sys_catalog.cc:564] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:16:45.460428 5203 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:16:45.464803 5210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.452812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54610 (local address 127.21.54.188:36963)
0811 17:16:45.452935 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:45.452943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:45.452971 (+ 28us) server_negotiation.cc:407] Connection header received
0811 17:16:45.453016 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:45.453019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:45.453070 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:45.453164 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:45.454194 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.456126 (+ 1932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.457221 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.457597 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.457702 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.459824 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:45.459828 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:45.459831 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:45.459858 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:45.461831 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.462438 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.462445 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.462450 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.462561 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.462948 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.462965 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.462970 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.463242 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:45.463882 (+ 640us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:45.464380 (+ 498us) server_negotiation.cc:299] Negotiation successful
0811 17:16:45.464570 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 17:16:45.467294 5243 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:16:45.467381 5243 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:16:45.470984 5223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } } peers { permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } } peers { permanent_uuid: "34ac20fc757d4dd594e245e74d3d7da3" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } } }
I20260811 17:16:45.471359 5223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [sys.catalog]: This master's current role is: FOLLOWER
Aug 11 17:16:45 dist-test-slave-fd7l krb5kdc[4931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468604, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for kudu/127.21.54.188@KRBTEST.COM
I20260811 17:16:45.464424 5034 master_runner.cc:186] Error getting master registration for 127.21.54.190:46259: OK, instance_id { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" instance_seqno: 1786468604927569 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 17:16:45.472369 5221 sys_catalog.cc:455] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } } peers { permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } } peers { permanent_uuid: "34ac20fc757d4dd594e245e74d3d7da3" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } } }
I20260811 17:16:45.472668 5221 sys_catalog.cc:458] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:16:45.473212 5034 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 17:16:45.477348 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.459090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:36963 (local address 127.0.0.1:54616)
0811 17:16:45.459425 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:45.459440 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:45.459550 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:45.460357 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:45.460364 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:45.460740 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:45.460889 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:45.460898 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.462127 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.462131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:45.468119 (+ 5988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:45.468134 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.468999 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.469005 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:45.469131 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.470333 (+ 1202us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:45.470356 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:45.472357 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:45.474760 (+ 2403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:45.474766 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:45.474773 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:45.475128 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:45.475799 (+ 671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:45.475803 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:45.475805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:45.475918 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:45.476717 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:45.476721 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:45.476846 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 17:16:45.476967 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 17:16:45.477154 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228}
I20260811 17:16:45.478145 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.459733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54616 (local address 127.21.54.188:36963)
0811 17:16:45.459860 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:45.459865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:45.459878 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:16:45.459915 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:45.459919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:45.459959 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:45.460042 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:45.461084 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.461988 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.468274 (+ 6286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.468695 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.468825 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.472586 (+ 3761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:45.472590 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:45.472593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:45.472621 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:45.474588 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.475529 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.475533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.475535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.475665 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.476215 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.476218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.476220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.476540 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:45.476652 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:45.477824 (+ 1172us) server_negotiation.cc:299] Negotiation successful
0811 17:16:45.477981 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:16:45.477341 5120 master_runner.cc:186] Error getting master registration for 127.21.54.190:46259: OK, instance_id { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" instance_seqno: 1786468604927569 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 17:16:45.480876 5120 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 17:16:45.481603 5119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.452691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37154 (local address 127.21.54.189:40533)
0811 17:16:45.453351 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:45.453356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:45.453372 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:16:45.453418 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:45.453423 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:45.453478 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:45.454472 (+ 994us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:45.454597 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.455507 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.457397 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.457773 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.457876 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.472940 (+ 15064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:45.472944 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:45.472947 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:45.472990 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:45.475360 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.476191 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.476195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.476199 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.476301 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.476713 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.476716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.476718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.476978 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:45.481219 (+ 4241us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:45.481310 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 17:16:45.481447 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570}
I20260811 17:16:45.482249 5027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774: Bootstrap starting.
I20260811 17:16:45.485242 5027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774: Neither blocks nor log segments found. Creating new log.
W20260811 17:16:45.486045 5249 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:16:45.486119 5249 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:16:45.486228 5027 log.cc:824] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774: Log is configured to *not* fsync() on all Append() calls
I20260811 17:16:45.487548 5030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:45.474594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36650 (local address 127.21.54.190:46259)
0811 17:16:45.474748 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:45.474754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:45.478023 (+ 3269us) server_negotiation.cc:407] Connection header received
0811 17:16:45.478071 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:45.478074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:45.478125 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:45.478210 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:45.479198 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.479977 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.481165 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:45.481495 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:45.481613 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:45.483519 (+ 1906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:45.483526 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:45.483535 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:45.483564 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:45.485447 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.486011 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.486015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.486017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.486117 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:45.486478 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:45.486482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:45.486483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:45.486729 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:45.486902 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:45.487229 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 17:16:45.487380 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 17:16:45.490656 5027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774: No bootstrap required, opened a new log
I20260811 17:16:45.495726 5027 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } } peers { permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } } peers { permanent_uuid: "34ac20fc757d4dd594e245e74d3d7da3" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } }
I20260811 17:16:45.496109 5027 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:16:45.496227 5027 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc764e78a99943f093f76dc58aeda774, State: Initialized, Role: FOLLOWER
I20260811 17:16:45.496925 5027 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } } peers { permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } } peers { permanent_uuid: "34ac20fc757d4dd594e245e74d3d7da3" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } }
I20260811 17:16:45.497630 5252 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } } peers { permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } } peers { permanent_uuid: "34ac20fc757d4dd594e245e74d3d7da3" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } } }
I20260811 17:16:45.497846 5252 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:16:45.498705 5027 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 17:16:45.506664 5264 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:16:45.506867 5264 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:16:45.509238 4947 master_runner.cc:263] No leader master found from master bc764e78a99943f093f76dc58aeda774
I20260811 17:16:45.509337 4947 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 17:16:45.529687 5223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:16:45.529925 5223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } } peers { permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } } peers { permanent_uuid: "34ac20fc757d4dd594e245e74d3d7da3" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } }
I20260811 17:16:45.531163 5223 leader_election.cc:288] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc764e78a99943f093f76dc58aeda774 (127.21.54.190:46259), f035bac3fc0e47a4ae2d049d16920e24 (127.21.54.189:40533)
I20260811 17:16:45.532024 5082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34ac20fc757d4dd594e245e74d3d7da3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f035bac3fc0e47a4ae2d049d16920e24" is_pre_election: true
I20260811 17:16:45.532410 5082 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34ac20fc757d4dd594e245e74d3d7da3 in term 0.
I20260811 17:16:45.533250 5144 leader_election.cc:302] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34ac20fc757d4dd594e245e74d3d7da3, f035bac3fc0e47a4ae2d049d16920e24; no voters:
I20260811 17:16:45.534519 4995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34ac20fc757d4dd594e245e74d3d7da3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc764e78a99943f093f76dc58aeda774" is_pre_election: true
I20260811 17:16:45.534914 4995 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34ac20fc757d4dd594e245e74d3d7da3 in term 0.
I20260811 17:16:45.535826 5223 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:16:45.535928 5223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:16:45.535989 5223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:16:45.540818 5223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } } peers { permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } } peers { permanent_uuid: "34ac20fc757d4dd594e245e74d3d7da3" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } }
I20260811 17:16:45.541992 5082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34ac20fc757d4dd594e245e74d3d7da3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f035bac3fc0e47a4ae2d049d16920e24"
I20260811 17:16:45.542169 5082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:16:45.543633 5082 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34ac20fc757d4dd594e245e74d3d7da3 in term 1.
I20260811 17:16:45.544085 5144 leader_election.cc:302] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34ac20fc757d4dd594e245e74d3d7da3, f035bac3fc0e47a4ae2d049d16920e24; no voters:
I20260811 17:16:45.544864 4995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34ac20fc757d4dd594e245e74d3d7da3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc764e78a99943f093f76dc58aeda774"
I20260811 17:16:45.545030 4995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:16:45.546568 4995 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34ac20fc757d4dd594e245e74d3d7da3 in term 1.
I20260811 17:16:45.548254 5223 leader_election.cc:288] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [CANDIDATE]: Term 1 election: Requested vote from peers bc764e78a99943f093f76dc58aeda774 (127.21.54.190:46259), f035bac3fc0e47a4ae2d049d16920e24 (127.21.54.189:40533)
I20260811 17:16:45.548502 5223 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:16:45.548683 5223 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [term 1 LEADER]: Becoming Leader. State: Replica: 34ac20fc757d4dd594e245e74d3d7da3, State: Running, Role: LEADER
I20260811 17:16:45.549139 5223 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } } peers { permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } } peers { permanent_uuid: "34ac20fc757d4dd594e245e74d3d7da3" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } }
I20260811 17:16:45.551474 5223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34ac20fc757d4dd594e245e74d3d7da3. Latest consensus state: current_term: 1 leader_uuid: "34ac20fc757d4dd594e245e74d3d7da3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } } peers { permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } } peers { permanent_uuid: "34ac20fc757d4dd594e245e74d3d7da3" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } } }
I20260811 17:16:45.551664 5223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [sys.catalog]: This master's current role is: LEADER
I20260811 17:16:45.552264 5269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:16:45.553457 5269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:16:45.565634 5082 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24 [term 1 FOLLOWER]: Refusing update from remote peer 34ac20fc757d4dd594e245e74d3d7da3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:16:45.566807 5267 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:16:45.571924 5221 sys_catalog.cc:455] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34ac20fc757d4dd594e245e74d3d7da3. Latest consensus state: current_term: 1 leader_uuid: "34ac20fc757d4dd594e245e74d3d7da3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } } peers { permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } } peers { permanent_uuid: "34ac20fc757d4dd594e245e74d3d7da3" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } } }
I20260811 17:16:45.572073 5221 sys_catalog.cc:458] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:16:45.576262 4995 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774 [term 1 FOLLOWER]: Refusing update from remote peer 34ac20fc757d4dd594e245e74d3d7da3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:16:45.577105 5267 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:16:45.582178 5252 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34ac20fc757d4dd594e245e74d3d7da3. Latest consensus state: current_term: 1 leader_uuid: "34ac20fc757d4dd594e245e74d3d7da3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } } peers { permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } } peers { permanent_uuid: "34ac20fc757d4dd594e245e74d3d7da3" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } } }
I20260811 17:16:45.582345 5252 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:16:45.592047 5267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "34ac20fc757d4dd594e245e74d3d7da3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } } peers { permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } } peers { permanent_uuid: "34ac20fc757d4dd594e245e74d3d7da3" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } } }
I20260811 17:16:45.592204 5267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [sys.catalog]: This master's current role is: LEADER
I20260811 17:16:45.593487 5252 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "34ac20fc757d4dd594e245e74d3d7da3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } } peers { permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } } peers { permanent_uuid: "34ac20fc757d4dd594e245e74d3d7da3" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } } }
I20260811 17:16:45.593616 5252 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:16:45.594113 5270 mvcc.cc:205] Tried to move back new op lower bound from 7317375408387846144 to 7317375408333688832. Current Snapshot: MvccSnapshot[applied={T|T < 7317375408387846144}]
I20260811 17:16:45.594923 5273 mvcc.cc:205] Tried to move back new op lower bound from 7317375408387846144 to 7317375408333688832. Current Snapshot: MvccSnapshot[applied={T|T < 7317375408387846144}]
I20260811 17:16:45.595794 5223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "34ac20fc757d4dd594e245e74d3d7da3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } } peers { permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } } peers { permanent_uuid: "34ac20fc757d4dd594e245e74d3d7da3" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } } }
I20260811 17:16:45.595916 5223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3 [sys.catalog]: This master's current role is: LEADER
I20260811 17:16:45.597175 5221 sys_catalog.cc:455] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "34ac20fc757d4dd594e245e74d3d7da3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc764e78a99943f093f76dc58aeda774" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46259 } } peers { permanent_uuid: "f035bac3fc0e47a4ae2d049d16920e24" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 40533 } } peers { permanent_uuid: "34ac20fc757d4dd594e245e74d3d7da3" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 36963 } } }
I20260811 17:16:45.597304 5221 sys_catalog.cc:458] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:16:45.605530 5269 catalog_manager.cc:1347] Generated new cluster ID: c59ce23691154fcfb00c4cbefb3faf69
I20260811 17:16:45.605607 5269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:16:45.618662 5271 mvcc.cc:205] Tried to move back new op lower bound from 7317375408387846144 to 7317375408333688832. Current Snapshot: MvccSnapshot[applied={T|T < 7317375408387846144}]
I20260811 17:16:45.639233 5269 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:16:45.640648 5269 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:16:45.655251 5269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34ac20fc757d4dd594e245e74d3d7da3: Generated new TSK 0
I20260811 17:16:45.656311 5269 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:16:46.468968 5243 catalog_manager.cc:1259] Loaded cluster ID: c59ce23691154fcfb00c4cbefb3faf69
I20260811 17:16:46.469060 5243 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24: loading cluster ID for follower catalog manager: success
I20260811 17:16:46.470819 5243 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24: acquiring CA information for follower catalog manager: success
I20260811 17:16:46.472070 5243 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f035bac3fc0e47a4ae2d049d16920e24: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 17:16:46.508308 5264 catalog_manager.cc:1259] Loaded cluster ID: c59ce23691154fcfb00c4cbefb3faf69
I20260811 17:16:46.508399 5264 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774: loading cluster ID for follower catalog manager: success
I20260811 17:16:46.510181 5264 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774: acquiring CA information for follower catalog manager: success
I20260811 17:16:46.511420 5264 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bc764e78a99943f093f76dc58aeda774: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 17:16:46.733419 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 4947
I20260811 17:16:46.744343 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 5034
I20260811 17:16:46.753275 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 5120
2026-08-11T17:16:46Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3296 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:16:46 dist-test-slave-fd7l krb5kdc[5285](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:16:46 dist-test-slave-fd7l krb5kdc[5285](info): set up 2 sockets
Aug 11 17:16:46 dist-test-slave-fd7l krb5kdc[5285](info): commencing operation
krb5kdc: starting...
W20260811 17:16:47.953307 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:16:48 dist-test-slave-fd7l krb5kdc[5285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:16:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:16:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:16:48.119971 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:39455
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:37743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:39455
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:16:48.245038 5306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:48.245337 5306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:48.245380 5306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:48.249797 5306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:16:48.249868 5306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:48.249889 5306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:48.249907 5306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:16:48.249923 5306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:16:48.255504 5306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:37743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:39455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:39455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:48.256740 5306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:48.257874 5306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:48.267050 5315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:48.267062 5312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:48.267134 5306 server_base.cc:1034] running on GCE node
W20260811 17:16:48.267059 5313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:48.267880 5306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:48.269063 5306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:48.270236 5306 hybrid_clock.cc:648] HybridClock initialized: now 1786468608270222 us; error 27 us; skew 500 ppm
Aug 11 17:16:48 dist-test-slave-fd7l krb5kdc[5285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468608, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:48.273124 5306 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:16:48.273980 5306 webserver.cc:460] Webserver started at http://127.21.54.190:37021/ using document root <none> and password file <none>
I20260811 17:16:48.274515 5306 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:48.274576 5306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:48.274767 5306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:16:48.276895 5306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "6ef83cff698842f282b5ef6521fc2edb"
format_stamp: "Formatted at 2026-08-11 17:16:48 on dist-test-slave-fd7l"
server_key: "6afe4ec0f6765c73e98a13d0576be089"
server_key_iv: "18ec9bb3b79f9c076eac8f9e67f2fc03"
server_key_version: "encryptionkey@0"
I20260811 17:16:48.277433 5306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "6ef83cff698842f282b5ef6521fc2edb"
format_stamp: "Formatted at 2026-08-11 17:16:48 on dist-test-slave-fd7l"
server_key: "6afe4ec0f6765c73e98a13d0576be089"
server_key_iv: "18ec9bb3b79f9c076eac8f9e67f2fc03"
server_key_version: "encryptionkey@0"
I20260811 17:16:48.281365 5306 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:16:48.283845 5323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:48.285279 5306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:16:48.285434 5306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "6ef83cff698842f282b5ef6521fc2edb"
format_stamp: "Formatted at 2026-08-11 17:16:48 on dist-test-slave-fd7l"
server_key: "6afe4ec0f6765c73e98a13d0576be089"
server_key_iv: "18ec9bb3b79f9c076eac8f9e67f2fc03"
server_key_version: "encryptionkey@0"
I20260811 17:16:48.285562 5306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:48.296312 5306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:48.299276 5306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:48.299494 5306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:48.307070 5381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:39455 every 8 connection(s)
I20260811 17:16:48.307081 5306 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:39455
I20260811 17:16:48.308236 5306 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:16:48.311256 5382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:16:48.317137 5382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb: Bootstrap starting.
I20260811 17:16:48.318396 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 5306
I20260811 17:16:48.318557 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:16:48.318858 21722 external_mini_cluster.cc:1428] Setting key 40d464eadc5c7659c3a039fa7d41caa3
I20260811 17:16:48.320158 5382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb: Neither blocks nor log segments found. Creating new log.
I20260811 17:16:48.321132 5382 log.cc:824] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb: Log is configured to *not* fsync() on all Append() calls
I20260811 17:16:48.323549 5382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb: No bootstrap required, opened a new log
Aug 11 17:16:48 dist-test-slave-fd7l krb5kdc[5285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:16:48.328457 5382 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef83cff698842f282b5ef6521fc2edb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39455 } }
I20260811 17:16:48.328841 5382 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:16:48.328930 5382 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ef83cff698842f282b5ef6521fc2edb, State: Initialized, Role: FOLLOWER
I20260811 17:16:48.329478 5382 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef83cff698842f282b5ef6521fc2edb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39455 } }
I20260811 17:16:48.329650 5382 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:16:48.329738 5382 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:16:48.329845 5382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:16:48.330847 5382 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef83cff698842f282b5ef6521fc2edb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39455 } }
I20260811 17:16:48.331230 5382 leader_election.cc:302] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ef83cff698842f282b5ef6521fc2edb; no voters:
I20260811 17:16:48.331629 5382 leader_election.cc:288] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:16:48.331704 5387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:16:48.331915 5387 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb [term 1 LEADER]: Becoming Leader. State: Replica: 6ef83cff698842f282b5ef6521fc2edb, State: Running, Role: LEADER
I20260811 17:16:48.332291 5387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef83cff698842f282b5ef6521fc2edb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39455 } }
I20260811 17:16:48.333118 5382 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:16:48.334331 5390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ef83cff698842f282b5ef6521fc2edb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef83cff698842f282b5ef6521fc2edb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39455 } } }
I20260811 17:16:48.334481 5390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb [sys.catalog]: This master's current role is: LEADER
I20260811 17:16:48.334812 5391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ef83cff698842f282b5ef6521fc2edb. Latest consensus state: current_term: 1 leader_uuid: "6ef83cff698842f282b5ef6521fc2edb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef83cff698842f282b5ef6521fc2edb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39455 } } }
I20260811 17:16:48.334968 5391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb [sys.catalog]: This master's current role is: LEADER
I20260811 17:16:48.335227 5385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:48.320167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41036 (local address 127.21.54.190:39455)
0811 17:16:48.320696 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:48.320712 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:48.320748 (+ 36us) server_negotiation.cc:407] Connection header received
0811 17:16:48.321512 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:48.321532 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:48.321860 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:48.322170 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:48.323157 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:48.324299 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:48.325547 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:48.326070 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:48.326205 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:48.329395 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:48.329401 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:48.329417 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:48.329448 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:48.331468 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:48.332088 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:48.332094 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:48.332100 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:48.332251 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:48.332615 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:48.332618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:48.332620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:48.333049 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:48.333287 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:48.333619 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 17:16:48.333951 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":156,"threads_started":1}
I20260811 17:16:48.335925 5398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:16:48.338482 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:16:48.339182 5398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:16:48.345080 5398 catalog_manager.cc:1347] Generated new cluster ID: f8d7f80809074ace9c18779957c9949c
I20260811 17:16:48.345157 5398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:16:48.356421 5398 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:16:48.357415 5398 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:16:48.366277 5398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ef83cff698842f282b5ef6521fc2edb: Generated new TSK 0
I20260811 17:16:48.367012 5398 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:16:48.394659 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 5306
2026-08-11T17:16:48Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1640 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:16:48 dist-test-slave-fd7l krb5kdc[5413](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:16:48 dist-test-slave-fd7l krb5kdc[5413](info): set up 2 sockets
Aug 11 17:16:48 dist-test-slave-fd7l krb5kdc[5413](info): commencing operation
krb5kdc: starting...
W20260811 17:16:49.575930 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:16:49 dist-test-slave-fd7l krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:16:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:16:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:16:49.711831 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:45313
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:45313
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:16:49.826188 5427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:49.826510 5427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:49.826583 5427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:49.830638 5427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:16:49.830727 5427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:49.830766 5427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:49.830823 5427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:16:49.830864 5427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:16:49.836145 5427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:45313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:45313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:49.837324 5427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:49.838364 5427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:49.847086 5432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:49.847088 5433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:49.847291 5427 server_base.cc:1034] running on GCE node
W20260811 17:16:49.847453 5435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:49.848001 5427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:49.849179 5427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:49.850345 5427 hybrid_clock.cc:648] HybridClock initialized: now 1786468609850332 us; error 26 us; skew 500 ppm
Aug 11 17:16:49 dist-test-slave-fd7l krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468609, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:49.853227 5427 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:16:49.854251 5427 webserver.cc:460] Webserver started at http://127.21.54.190:42893/ using document root <none> and password file <none>
I20260811 17:16:49.854842 5427 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:49.854934 5427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:49.855182 5427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:16:49.857314 5427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d"
format_stamp: "Formatted at 2026-08-11 17:16:49 on dist-test-slave-fd7l"
server_key: "d0941b49ecbca6bf43dada315016e43c"
server_key_iv: "2ab20b81920d9dd74c8cf22520ce03c6"
server_key_version: "encryptionkey@0"
I20260811 17:16:49.857903 5427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d"
format_stamp: "Formatted at 2026-08-11 17:16:49 on dist-test-slave-fd7l"
server_key: "d0941b49ecbca6bf43dada315016e43c"
server_key_iv: "2ab20b81920d9dd74c8cf22520ce03c6"
server_key_version: "encryptionkey@0"
I20260811 17:16:49.861693 5427 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 17:16:49.864109 5446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:49.865404 5427 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 17:16:49.865551 5427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d"
format_stamp: "Formatted at 2026-08-11 17:16:49 on dist-test-slave-fd7l"
server_key: "d0941b49ecbca6bf43dada315016e43c"
server_key_iv: "2ab20b81920d9dd74c8cf22520ce03c6"
server_key_version: "encryptionkey@0"
I20260811 17:16:49.865698 5427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:49.888072 5427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:49.892134 5427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:49.892385 5427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:49.900818 5427 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:45313
I20260811 17:16:49.901139 5503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:45313 every 8 connection(s)
I20260811 17:16:49.902012 5427 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:16:49.905049 5504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:16:49.909186 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 5427
I20260811 17:16:49.909346 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:16:49.909715 21722 external_mini_cluster.cc:1428] Setting key fabe3163c6968c9569f0f01b7a3cce16
I20260811 17:16:49.911767 5504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d: Bootstrap starting.
I20260811 17:16:49.914106 5504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d: Neither blocks nor log segments found. Creating new log.
I20260811 17:16:49.914851 5504 log.cc:824] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d: Log is configured to *not* fsync() on all Append() calls
I20260811 17:16:49.916909 5504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d: No bootstrap required, opened a new log
Aug 11 17:16:49 dist-test-slave-fd7l krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:16:49.920753 5504 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45313 } }
I20260811 17:16:49.921015 5504 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:16:49.921114 5504 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc29907ebf7c4c7a8f23ebbe05ebb96d, State: Initialized, Role: FOLLOWER
I20260811 17:16:49.921545 5504 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45313 } }
I20260811 17:16:49.921698 5504 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:16:49.921772 5504 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:16:49.921871 5504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:16:49.922765 5504 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45313 } }
I20260811 17:16:49.923118 5504 leader_election.cc:302] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc29907ebf7c4c7a8f23ebbe05ebb96d; no voters:
I20260811 17:16:49.923430 5504 leader_election.cc:288] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:16:49.923542 5509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:16:49.923830 5509 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 1 LEADER]: Becoming Leader. State: Replica: dc29907ebf7c4c7a8f23ebbe05ebb96d, State: Running, Role: LEADER
I20260811 17:16:49.924166 5509 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45313 } }
I20260811 17:16:49.924762 5504 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:16:49.925457 5507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:49.911279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50236 (local address 127.21.54.190:45313)
0811 17:16:49.911763 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:49.911778 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:49.911815 (+ 37us) server_negotiation.cc:407] Connection header received
0811 17:16:49.912500 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:49.912592 (+ 92us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:49.912917 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:49.913260 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:49.914079 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:49.915234 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:49.916068 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:49.916620 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:49.916825 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:49.919308 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:49.919321 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:49.919333 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:49.919360 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:49.921565 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:49.922171 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:49.922177 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:49.922182 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:49.922299 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:49.922623 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:49.922627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:49.922629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:49.923108 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:49.923318 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:49.924056 (+ 738us) server_negotiation.cc:299] Negotiation successful
0811 17:16:49.924394 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":143,"threads_started":1}
I20260811 17:16:49.926000 5512 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc29907ebf7c4c7a8f23ebbe05ebb96d. Latest consensus state: current_term: 1 leader_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45313 } } }
I20260811 17:16:49.926115 5512 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [sys.catalog]: This master's current role is: LEADER
I20260811 17:16:49.926443 5510 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45313 } } }
I20260811 17:16:49.926560 5510 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [sys.catalog]: This master's current role is: LEADER
I20260811 17:16:49.926553 5520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:16:49.930059 5520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:16:49.936085 5520 catalog_manager.cc:1347] Generated new cluster ID: 8e7bb94620e0442fbd9f4fd61801b4a2
I20260811 17:16:49.936156 5520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:16:49.946872 5520 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:16:49.947839 5520 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:16:49.957818 5520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d: Generated new TSK 0
I20260811 17:16:49.958598 5520 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
I20260811 17:16:49.992988 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:45313
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:16:50.109319 5535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:50.109607 5535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:50.109653 5535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:50.114400 5535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:50.114470 5535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:50.114533 5535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:16:50.120110 5535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:45313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:50.121374 5535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:50.122403 5535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:50.133314 5540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:50.133318 5544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:50.133314 5542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:50.133723 5535 server_base.cc:1034] running on GCE node
I20260811 17:16:50.134125 5535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:50.134819 5535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:50.136023 5535 hybrid_clock.cc:648] HybridClock initialized: now 1786468610135991 us; error 49 us; skew 500 ppm
Aug 11 17:16:50 dist-test-slave-fd7l krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468610, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:50.138988 5535 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:16:50.139983 5535 webserver.cc:460] Webserver started at http://127.21.54.129:39869/ using document root <none> and password file <none>
I20260811 17:16:50.140621 5535 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:50.140714 5535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:50.140966 5535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:16:50.143129 5535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "da6b2e8a85c64c859170faef63149240"
format_stamp: "Formatted at 2026-08-11 17:16:50 on dist-test-slave-fd7l"
server_key: "b6987468bdddf967cddea6347243c39f"
server_key_iv: "f6ca41454b63fa28ca5ed1d650aac463"
server_key_version: "encryptionkey@0"
I20260811 17:16:50.143735 5535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "da6b2e8a85c64c859170faef63149240"
format_stamp: "Formatted at 2026-08-11 17:16:50 on dist-test-slave-fd7l"
server_key: "b6987468bdddf967cddea6347243c39f"
server_key_iv: "f6ca41454b63fa28ca5ed1d650aac463"
server_key_version: "encryptionkey@0"
I20260811 17:16:50.147931 5535 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:16:50.150599 5551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:50.152202 5535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:16:50.152318 5535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "da6b2e8a85c64c859170faef63149240"
format_stamp: "Formatted at 2026-08-11 17:16:50 on dist-test-slave-fd7l"
server_key: "b6987468bdddf967cddea6347243c39f"
server_key_iv: "f6ca41454b63fa28ca5ed1d650aac463"
server_key_version: "encryptionkey@0"
I20260811 17:16:50.152427 5535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:50.170258 5535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:50.173991 5535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:50.174178 5535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:50.174873 5535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:16:50.175750 5535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:16:50.175799 5535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:50.175848 5535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:16:50.175864 5535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:50.185566 5535 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:37467
I20260811 17:16:50.185590 5685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:37467 every 8 connection(s)
I20260811 17:16:50.186560 5535 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:16:50.190963 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 5535
I20260811 17:16:50.191138 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:16:50.191450 21722 external_mini_cluster.cc:1428] Setting key 9cb25e4297f7d34de7f48c1e5869e9b5
Aug 11 17:16:50 dist-test-slave-fd7l krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468610, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:16:50.202716 5507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:50.188563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:44393 (local address 127.21.54.190:45313)
0811 17:16:50.188725 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:50.188729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:50.189633 (+ 904us) server_negotiation.cc:407] Connection header received
0811 17:16:50.191038 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:50.191042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:50.191121 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:50.191227 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:50.192738 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.193483 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.194665 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.195067 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.195196 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:50.198321 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:50.198328 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:50.198335 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:50.198375 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:50.200267 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:50.200914 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:50.200921 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:50.200924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:50.201029 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:50.201405 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:50.201409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:50.201411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:50.201681 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:50.201798 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:50.202351 (+ 553us) server_negotiation.cc:299] Negotiation successful
0811 17:16:50.202501 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
WARNING: no policy specified for kudu/127.21.54.130@KRBTEST.COM; defaulting to no policy
I20260811 17:16:50.203545 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:50.188969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:44393)
0811 17:16:50.189445 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:50.189488 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:50.190817 (+ 1329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:50.191409 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:50.191417 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:50.191877 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:50.192581 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:50.192602 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.193642 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.193645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:50.194528 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:50.194537 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.195277 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.195283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:50.195513 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:50.196270 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:50.196295 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:50.198107 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:50.200401 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:50.200406 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:50.200422 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:50.200788 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:50.201138 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:50.201140 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:50.201141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:50.201294 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:50.201796 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:50.201802 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:50.202059 (+ 257us) client_negotiation.cc:769] Sending connection context
0811 17:16:50.202239 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 17:16:50.202520 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":122,"threads_started":1}
I20260811 17:16:50.205000 5686 heartbeater.cc:347] Connected to a master server at 127.21.54.190:45313
I20260811 17:16:50.205248 5686 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:50.205811 5686 heartbeater.cc:511] Master 127.21.54.190:45313 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.130@KRBTEST.COM" created.
I20260811 17:16:50.207655 5464 ts_manager.cc:194] Registered new tserver with Master: da6b2e8a85c64c859170faef63149240 (127.21.54.129:37467)
I20260811 17:16:50.209178 5464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:44393
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
I20260811 17:16:50.224491 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:45313
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:16:50.341472 5693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:50.341797 5693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:50.341871 5693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:50.346016 5693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:50.346115 5693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:50.346206 5693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:16:50.351850 5693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:45313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:50.353140 5693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:50.354225 5693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:50.364439 5700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:50.364581 5703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:50.364812 5693 server_base.cc:1034] running on GCE node
W20260811 17:16:50.364475 5699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:50.365444 5693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:50.366369 5693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:50.367602 5693 hybrid_clock.cc:648] HybridClock initialized: now 1786468610367564 us; error 52 us; skew 500 ppm
Aug 11 17:16:50 dist-test-slave-fd7l krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468610, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:50.370899 5693 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:16:50.371886 5693 webserver.cc:460] Webserver started at http://127.21.54.130:40723/ using document root <none> and password file <none>
I20260811 17:16:50.372676 5693 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:50.372774 5693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:50.373015 5693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:16:50.375331 5693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "8692e983abb54c31bced385540a68fee"
format_stamp: "Formatted at 2026-08-11 17:16:50 on dist-test-slave-fd7l"
server_key: "b32d048c1a92ffc7c7eb2c07b47a031a"
server_key_iv: "cb5149d50e779e342b17080b44540df6"
server_key_version: "encryptionkey@0"
I20260811 17:16:50.375942 5693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "8692e983abb54c31bced385540a68fee"
format_stamp: "Formatted at 2026-08-11 17:16:50 on dist-test-slave-fd7l"
server_key: "b32d048c1a92ffc7c7eb2c07b47a031a"
server_key_iv: "cb5149d50e779e342b17080b44540df6"
server_key_version: "encryptionkey@0"
I20260811 17:16:50.379886 5693 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:16:50.382588 5710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:50.383908 5693 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 17:16:50.384038 5693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "8692e983abb54c31bced385540a68fee"
format_stamp: "Formatted at 2026-08-11 17:16:50 on dist-test-slave-fd7l"
server_key: "b32d048c1a92ffc7c7eb2c07b47a031a"
server_key_iv: "cb5149d50e779e342b17080b44540df6"
server_key_version: "encryptionkey@0"
I20260811 17:16:50.384168 5693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:50.402763 5693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:50.406133 5693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:50.406355 5693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:50.406994 5693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:16:50.407859 5693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:16:50.407907 5693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:50.407949 5693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:16:50.407964 5693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:50.417660 5693 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:45421
I20260811 17:16:50.417694 5843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:45421 every 8 connection(s)
I20260811 17:16:50.418612 5693 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:16:50.423337 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 5693
I20260811 17:16:50.423482 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:16:50.423849 21722 external_mini_cluster.cc:1428] Setting key 99072ea630b8d5ededc1062d9e502930
Aug 11 17:16:50 dist-test-slave-fd7l krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468610, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:16:50.435636 5507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:50.420587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:52015 (local address 127.21.54.190:45313)
0811 17:16:50.420759 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:50.420764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:50.421746 (+ 982us) server_negotiation.cc:407] Connection header received
0811 17:16:50.423356 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:50.423362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:50.423431 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:50.423536 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:50.425279 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.426075 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.427544 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.427952 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.428098 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:50.431472 (+ 3374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:50.431475 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:50.431478 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:50.431505 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:50.433060 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:50.433772 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:50.433774 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:50.433776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:50.433853 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:50.434302 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:50.434305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:50.434306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:50.434504 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:50.434585 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:50.435361 (+ 776us) server_negotiation.cc:299] Negotiation successful
0811 17:16:50.435493 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
WARNING: no policy specified for kudu/127.21.54.131@KRBTEST.COM; defaulting to no policy
I20260811 17:16:50.436808 5846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:50.421047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:52015)
0811 17:16:50.421561 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:50.421612 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:50.423099 (+ 1487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:50.423769 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:50.423777 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:50.424247 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:50.425127 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:50.425149 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.426233 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.426236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:50.427400 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:50.427412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.428297 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.428301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:50.428563 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:50.429275 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:50.429294 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:50.431266 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:50.433192 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:50.433201 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:50.433223 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:50.433652 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:50.433952 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:50.433958 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:50.433962 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:50.434186 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:50.434633 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:50.434640 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:50.435038 (+ 398us) client_negotiation.cc:769] Sending connection context
0811 17:16:50.435302 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 17:16:50.435726 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":108,"threads_started":1}
I20260811 17:16:50.438354 5844 heartbeater.cc:347] Connected to a master server at 127.21.54.190:45313
I20260811 17:16:50.438606 5844 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.131@KRBTEST.COM" created.
I20260811 17:16:50.439260 5844 heartbeater.cc:511] Master 127.21.54.190:45313 requested a full tablet report, sending...
I20260811 17:16:50.440558 5464 ts_manager.cc:194] Registered new tserver with Master: 8692e983abb54c31bced385540a68fee (127.21.54.130:45421)
I20260811 17:16:50.441262 5464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:52015
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
I20260811 17:16:50.459540 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:45313
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:16:50.576812 5850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:50.577083 5850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:50.577128 5850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:50.581182 5850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:50.581245 5850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:50.581344 5850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:16:50.586917 5850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:45313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:50.588171 5850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:50.589365 5850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:50.599720 5856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:50.599773 5858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:50.599658 5855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:50.599880 5850 server_base.cc:1034] running on GCE node
I20260811 17:16:50.600361 5850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:50.601141 5850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:50.602327 5850 hybrid_clock.cc:648] HybridClock initialized: now 1786468610602340 us; error 56 us; skew 500 ppm
Aug 11 17:16:50 dist-test-slave-fd7l krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468610, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:50.605475 5850 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:16:50.606505 5850 webserver.cc:460] Webserver started at http://127.21.54.131:40469/ using document root <none> and password file <none>
I20260811 17:16:50.607112 5850 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:50.607180 5850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:50.607470 5850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:16:50.609663 5850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "d0e1c0675ae649f88adbeb4f0936fa23"
format_stamp: "Formatted at 2026-08-11 17:16:50 on dist-test-slave-fd7l"
server_key: "b50e7419b2977babf80ee9dad1ce8731"
server_key_iv: "d1f7db4d4266afd6997bf46e55710e53"
server_key_version: "encryptionkey@0"
I20260811 17:16:50.610255 5850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "d0e1c0675ae649f88adbeb4f0936fa23"
format_stamp: "Formatted at 2026-08-11 17:16:50 on dist-test-slave-fd7l"
server_key: "b50e7419b2977babf80ee9dad1ce8731"
server_key_iv: "d1f7db4d4266afd6997bf46e55710e53"
server_key_version: "encryptionkey@0"
I20260811 17:16:50.614192 5850 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:16:50.616904 5866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:50.618309 5850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:16:50.618477 5850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "d0e1c0675ae649f88adbeb4f0936fa23"
format_stamp: "Formatted at 2026-08-11 17:16:50 on dist-test-slave-fd7l"
server_key: "b50e7419b2977babf80ee9dad1ce8731"
server_key_iv: "d1f7db4d4266afd6997bf46e55710e53"
server_key_version: "encryptionkey@0"
I20260811 17:16:50.618603 5850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:50.638123 5850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:50.641966 5850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:50.642246 5850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:50.642920 5850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:16:50.643844 5850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:16:50.643919 5850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:50.643998 5850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:16:50.644042 5850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:50.654291 5850 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:38839
I20260811 17:16:50.654320 5995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:38839 every 8 connection(s)
I20260811 17:16:50.655241 5850 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:16:50.657979 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 5850
I20260811 17:16:50.658133 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:16:50.658499 21722 external_mini_cluster.cc:1428] Setting key 9f245e3398bd5181d224c3f0fbe4ad1b
Aug 11 17:16:50 dist-test-slave-fd7l krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468610, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:16:50.670332 5507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:50.657433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:38345 (local address 127.21.54.190:45313)
0811 17:16:50.657604 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:50.657610 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:50.658785 (+ 1175us) server_negotiation.cc:407] Connection header received
0811 17:16:50.660450 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:50.660453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:50.660529 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:50.660610 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:50.661800 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.662361 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.663533 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.663764 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.663884 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:50.666642 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:50.666645 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:50.666647 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:50.666676 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:50.668213 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:50.668803 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:50.668806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:50.668807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:50.668888 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:50.669243 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:50.669245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:50.669247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:50.669445 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:50.669526 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:50.670070 (+ 544us) server_negotiation.cc:299] Negotiation successful
0811 17:16:50.670201 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 17:16:50.671236 5998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:50.658055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:38345)
0811 17:16:50.658637 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:50.658681 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:50.660247 (+ 1566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:50.660782 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:50.660789 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:50.661150 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:50.661650 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:50.661664 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.662503 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.662507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:50.663398 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:50.663408 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.663930 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.663934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:50.664120 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:50.664828 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:50.664845 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:50.666432 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:50.668330 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:50.668335 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:50.668349 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:50.668710 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:50.668976 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:50.668978 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:50.668980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:50.669136 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:50.669551 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:50.669558 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:50.669842 (+ 284us) client_negotiation.cc:769] Sending connection context
0811 17:16:50.670011 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 17:16:50.670346 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":162,"threads_started":1}
I20260811 17:16:50.672700 5996 heartbeater.cc:347] Connected to a master server at 127.21.54.190:45313
I20260811 17:16:50.672994 5996 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:50.673519 5996 heartbeater.cc:511] Master 127.21.54.190:45313 requested a full tablet report, sending...
I20260811 17:16:50.674525 5464 ts_manager.cc:194] Registered new tserver with Master: d0e1c0675ae649f88adbeb4f0936fa23 (127.21.54.131:38839)
I20260811 17:16:50.675105 5464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:38345
I20260811 17:16:50.685319 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:16:50.696202 5507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:50.687651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50244 (local address 127.21.54.190:45313)
0811 17:16:50.687808 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:50.687813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:50.688034 (+ 221us) server_negotiation.cc:407] Connection header received
0811 17:16:50.688161 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:50.688164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:50.688211 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:50.688285 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:50.689280 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.689796 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.690657 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.690885 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.691021 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:50.693040 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:50.693044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:50.693046 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:50.693071 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:50.694609 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:50.695032 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:50.695035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:50.695036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:50.695130 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:50.695371 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:50.695373 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:50.695375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:50.695606 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:50.695715 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:50.695904 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 17:16:50.696003 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 17:16:50.699234 5464 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50244:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:16:50.702461 5464 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:16:50.717397 6007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:50.711073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:38839 (local address 127.0.0.1:44648)
0811 17:16:50.711739 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:50.711790 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:50.711922 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:50.712819 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:50.712825 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:50.712855 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:16:50.713027 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:50.713034 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.714263 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.714268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:50.715541 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:50.715563 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.716701 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.716705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:50.716822 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:50.716965 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 17:16:50.717088 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 17:16:50.717214 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":47,"thread_start_us":99,"threads_started":1}
I20260811 17:16:50.717586 6011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:50.711510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:45421 (local address 127.0.0.1:35532)
0811 17:16:50.712071 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:50.712085 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:50.712187 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:50.712944 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:50.712947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:50.712961 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:16:50.713041 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:50.713046 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.714677 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.714681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:50.716113 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:50.716123 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.717139 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.717143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:50.717254 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:50.717279 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 17:16:50.717359 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 17:16:50.717430 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":2,"thread_start_us":124,"threads_started":1}
I20260811 17:16:50.717607 6010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:50.711798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:37467 (local address 127.0.0.1:46072)
0811 17:16:50.713112 (+ 1314us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:50.713128 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:50.713232 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:50.713682 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:50.713685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:50.713705 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:16:50.713802 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:50.713808 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.715020 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.715027 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:50.715928 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:50.715939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.717281 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.717284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:50.717376 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:50.717394 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 17:16:50.717455 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 17:16:50.717510 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1249,"mutex_wait_us":2,"thread_start_us":110,"threads_started":1}
I20260811 17:16:50.717890 6012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:50.711989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35532 (local address 127.21.54.130:45421)
0811 17:16:50.712435 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:50.712444 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:50.712484 (+ 40us) server_negotiation.cc:407] Connection header received
0811 17:16:50.712581 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:50.712587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:50.712787 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:50.713228 (+ 441us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:16:50.713404 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.714532 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.716234 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.716991 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.717107 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:50.717238 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:50.717404 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 17:16:50.717724 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":142,"threads_started":1}
I20260811 17:16:50.717965 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:50.712349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46072 (local address 127.21.54.129:37467)
0811 17:16:50.712721 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:50.712727 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:50.713233 (+ 506us) server_negotiation.cc:407] Connection header received
0811 17:16:50.713345 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:50.713351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:50.713528 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:50.713686 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:16:50.713924 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.714910 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.716072 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.716938 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.717069 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:50.717223 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:50.717676 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 17:16:50.717792 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":91,"threads_started":1}
I20260811 17:16:50.718318 6009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:50.711180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44648 (local address 127.21.54.131:38839)
0811 17:16:50.712245 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:50.712260 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:50.712289 (+ 29us) server_negotiation.cc:407] Connection header received
0811 17:16:50.712367 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:50.712372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:50.712624 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:50.712825 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:16:50.713166 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.714139 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.715693 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.716565 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.716689 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:50.716842 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:50.718064 (+ 1222us) server_negotiation.cc:299] Negotiation successful
0811 17:16:50.718175 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":949,"thread_start_us":109,"threads_started":1}
I20260811 17:16:50.720822 5608 tablet_service.cc:1467] Processing CreateTablet for tablet 91dc3c3d685b45dc9dbbaab6b377f63d (DEFAULT_TABLE table=test-table [id=214f099180c54e39b26e845e7371979f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:16:50.720824 5923 tablet_service.cc:1467] Processing CreateTablet for tablet 91dc3c3d685b45dc9dbbaab6b377f63d (DEFAULT_TABLE table=test-table [id=214f099180c54e39b26e845e7371979f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:16:50.720811 5765 tablet_service.cc:1467] Processing CreateTablet for tablet 91dc3c3d685b45dc9dbbaab6b377f63d (DEFAULT_TABLE table=test-table [id=214f099180c54e39b26e845e7371979f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:16:50.721964 5608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91dc3c3d685b45dc9dbbaab6b377f63d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:16:50.721966 5923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91dc3c3d685b45dc9dbbaab6b377f63d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:16:50.721966 5765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91dc3c3d685b45dc9dbbaab6b377f63d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:16:50.729188 6014 tablet_bootstrap.cc:492] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee: Bootstrap starting.
I20260811 17:16:50.730157 6015 tablet_bootstrap.cc:492] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240: Bootstrap starting.
I20260811 17:16:50.731899 6014 tablet_bootstrap.cc:654] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee: Neither blocks nor log segments found. Creating new log.
I20260811 17:16:50.731938 6015 tablet_bootstrap.cc:654] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240: Neither blocks nor log segments found. Creating new log.
I20260811 17:16:50.732762 6015 log.cc:824] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240: Log is configured to *not* fsync() on all Append() calls
I20260811 17:16:50.732782 6014 log.cc:824] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee: Log is configured to *not* fsync() on all Append() calls
I20260811 17:16:50.732860 6016 tablet_bootstrap.cc:492] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23: Bootstrap starting.
I20260811 17:16:50.734768 6016 tablet_bootstrap.cc:654] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23: Neither blocks nor log segments found. Creating new log.
I20260811 17:16:50.734916 6015 tablet_bootstrap.cc:492] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240: No bootstrap required, opened a new log
I20260811 17:16:50.735069 6014 tablet_bootstrap.cc:492] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee: No bootstrap required, opened a new log
I20260811 17:16:50.735131 6015 ts_tablet_manager.cc:1397] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 17:16:50.735232 6014 ts_tablet_manager.cc:1397] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 17:16:50.735957 6016 log.cc:824] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23: Log is configured to *not* fsync() on all Append() calls
I20260811 17:16:50.737702 6016 tablet_bootstrap.cc:492] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23: No bootstrap required, opened a new log
I20260811 17:16:50.737905 6016 ts_tablet_manager.cc:1397] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 17:16:50.739265 6014 raft_consensus.cc:348] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } }
I20260811 17:16:50.739562 6014 raft_consensus.cc:374] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:16:50.739624 6014 raft_consensus.cc:729] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8692e983abb54c31bced385540a68fee, State: Initialized, Role: FOLLOWER
I20260811 17:16:50.739621 6015 raft_consensus.cc:348] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } }
I20260811 17:16:50.739852 6015 raft_consensus.cc:374] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:16:50.739900 6015 raft_consensus.cc:729] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da6b2e8a85c64c859170faef63149240, State: Initialized, Role: FOLLOWER
I20260811 17:16:50.740145 6014 consensus_queue.cc:260] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } }
I20260811 17:16:50.740929 5844 heartbeater.cc:503] Master 127.21.54.190:45313 was elected leader, sending a full tablet report...
I20260811 17:16:50.740886 6015 consensus_queue.cc:260] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } }
I20260811 17:16:50.741258 6014 ts_tablet_manager.cc:1428] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260811 17:16:50.741603 5686 heartbeater.cc:503] Master 127.21.54.190:45313 was elected leader, sending a full tablet report...
I20260811 17:16:50.741827 6015 ts_tablet_manager.cc:1428] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 17:16:50.742410 6016 raft_consensus.cc:348] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } }
I20260811 17:16:50.742656 6016 raft_consensus.cc:374] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:16:50.742713 6016 raft_consensus.cc:729] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e1c0675ae649f88adbeb4f0936fa23, State: Initialized, Role: FOLLOWER
I20260811 17:16:50.743134 6016 consensus_queue.cc:260] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } }
I20260811 17:16:50.743731 5996 heartbeater.cc:503] Master 127.21.54.190:45313 was elected leader, sending a full tablet report...
I20260811 17:16:50.744068 6016 ts_tablet_manager.cc:1428] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260811 17:16:50.906255 5997 tablet.cc:2369] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:16:50.919934 5845 tablet.cc:2369] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:16:50.937999 5687 tablet.cc:2369] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:16:50.941982 6020 raft_consensus.cc:482] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:16:50.942346 6020 raft_consensus.cc:504] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } }
I20260811 17:16:50.943571 6020 leader_election.cc:288] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0e1c0675ae649f88adbeb4f0936fa23 (127.21.54.131:38839), da6b2e8a85c64c859170faef63149240 (127.21.54.129:37467)
I20260811 17:16:50.948370 6009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:50.944011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:36969 (local address 127.21.54.131:38839)
0811 17:16:50.944183 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:50.944188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:50.944345 (+ 157us) server_negotiation.cc:407] Connection header received
0811 17:16:50.944714 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:50.944717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:50.944785 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:50.944866 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:16:50.945286 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.945886 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.947186 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.947776 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.947880 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:50.947924 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:50.948171 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 17:16:50.948230 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 17:16:50.948396 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:50.943929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:38839 (local address 127.21.54.130:36969)
0811 17:16:50.944250 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:50.944265 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:50.944440 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:50.944907 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:50.944910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:50.944930 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:16:50.945092 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:50.945099 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.946029 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.946033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:50.947040 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:50.947052 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.947909 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.947913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:50.948029 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:50.948058 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 17:16:50.948148 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 17:16:50.948222 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":115,"threads_started":1}
I20260811 17:16:50.948939 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:50.944493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:35929 (local address 127.21.54.129:37467)
0811 17:16:50.944664 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:50.944670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:50.944829 (+ 159us) server_negotiation.cc:407] Connection header received
0811 17:16:50.945038 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:50.945044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:50.945100 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:50.945154 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:16:50.945574 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.946254 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.947672 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.948276 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.948392 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:50.948451 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:50.948742 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 17:16:50.948827 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 17:16:50.948949 6025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:50.944328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:37467 (local address 127.21.54.130:35929)
0811 17:16:50.944718 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:50.944732 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:50.944846 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:50.945298 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:50.945300 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:50.945317 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:16:50.945433 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:50.945439 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.946402 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.946405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:50.947391 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:50.947403 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:50.948419 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:50.948425 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:50.948561 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:50.948591 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 17:16:50.948683 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 17:16:50.948773 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":68,"threads_started":1}
I20260811 17:16:50.949323 5945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91dc3c3d685b45dc9dbbaab6b377f63d" candidate_uuid: "8692e983abb54c31bced385540a68fee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" is_pre_election: true
I20260811 17:16:50.949698 5945 raft_consensus.cc:2454] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8692e983abb54c31bced385540a68fee in term 0.
I20260811 17:16:50.949792 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91dc3c3d685b45dc9dbbaab6b377f63d" candidate_uuid: "8692e983abb54c31bced385540a68fee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da6b2e8a85c64c859170faef63149240" is_pre_election: true
I20260811 17:16:50.950122 5629 raft_consensus.cc:2454] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8692e983abb54c31bced385540a68fee in term 0.
I20260811 17:16:50.950214 5714 leader_election.cc:302] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8692e983abb54c31bced385540a68fee, d0e1c0675ae649f88adbeb4f0936fa23; no voters:
I20260811 17:16:50.950506 6020 raft_consensus.cc:2781] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:16:50.950618 6020 raft_consensus.cc:482] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:16:50.950687 6020 raft_consensus.cc:3037] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:16:50.951850 6020 raft_consensus.cc:504] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } }
I20260811 17:16:50.952239 6020 leader_election.cc:288] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [CANDIDATE]: Term 1 election: Requested vote from peers d0e1c0675ae649f88adbeb4f0936fa23 (127.21.54.131:38839), da6b2e8a85c64c859170faef63149240 (127.21.54.129:37467)
I20260811 17:16:50.952656 5945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91dc3c3d685b45dc9dbbaab6b377f63d" candidate_uuid: "8692e983abb54c31bced385540a68fee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e1c0675ae649f88adbeb4f0936fa23"
I20260811 17:16:50.952788 5629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91dc3c3d685b45dc9dbbaab6b377f63d" candidate_uuid: "8692e983abb54c31bced385540a68fee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da6b2e8a85c64c859170faef63149240"
I20260811 17:16:50.952836 5945 raft_consensus.cc:3037] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:16:50.952895 5629 raft_consensus.cc:3037] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:16:50.953924 5945 raft_consensus.cc:2454] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8692e983abb54c31bced385540a68fee in term 1.
I20260811 17:16:50.953927 5629 raft_consensus.cc:2454] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8692e983abb54c31bced385540a68fee in term 1.
I20260811 17:16:50.954236 5714 leader_election.cc:302] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8692e983abb54c31bced385540a68fee, da6b2e8a85c64c859170faef63149240; no voters:
I20260811 17:16:50.954419 6020 raft_consensus.cc:2781] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:16:50.954664 6020 raft_consensus.cc:686] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [term 1 LEADER]: Becoming Leader. State: Replica: 8692e983abb54c31bced385540a68fee, State: Running, Role: LEADER
I20260811 17:16:50.955085 6020 consensus_queue.cc:237] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } }
I20260811 17:16:50.958554 5463 catalog_manager.cc:5495] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee reported cstate change: term changed from 0 to 1, leader changed from <none> to 8692e983abb54c31bced385540a68fee (127.21.54.130). New cstate: current_term: 1 leader_uuid: "8692e983abb54c31bced385540a68fee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } health_report { overall_health: HEALTHY } } }
I20260811 17:16:51.011636 6012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:51.007921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35534 (local address 127.21.54.130:45421)
0811 17:16:51.008074 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:51.008079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:51.008096 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:16:51.008219 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:51.008223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:51.008291 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:51.008410 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:16:51.008710 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.009229 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.010086 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.010324 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.010435 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:51.010766 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:16:51.011211 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:16:51.011348 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:51.011409 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 17:16:51.011468 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 17:16:51.018335 5945 raft_consensus.cc:1261] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23 [term 1 FOLLOWER]: Refusing update from remote peer 8692e983abb54c31bced385540a68fee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:16:51.018335 5629 raft_consensus.cc:1261] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Refusing update from remote peer 8692e983abb54c31bced385540a68fee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:16:51.019258 6026 consensus_queue.cc:1035] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [LEADER]: Connected to new peer: Peer: permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:16:51.019546 6020 consensus_queue.cc:1035] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:16:51.029347 6030 mvcc.cc:205] Tried to move back new op lower bound from 7317375430721753088 to 7317375430475083776. Current Snapshot: MvccSnapshot[applied={T|T < 7317375430721753088}]
I20260811 17:16:51.030264 6031 mvcc.cc:205] Tried to move back new op lower bound from 7317375430721753088 to 7317375430475083776. Current Snapshot: MvccSnapshot[applied={T|T < 7317375430721753088}]
I20260811 17:16:51.038786 5464 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:16:51.038985 5464 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:16:51.041841 5464 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d: Sending DeleteTablet for 3 replicas of tablet 91dc3c3d685b45dc9dbbaab6b377f63d
I20260811 17:16:51.042929 5765 tablet_service.cc:1514] Processing DeleteTablet for tablet 91dc3c3d685b45dc9dbbaab6b377f63d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:16:51 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:35532
I20260811 17:16:51.042932 5923 tablet_service.cc:1514] Processing DeleteTablet for tablet 91dc3c3d685b45dc9dbbaab6b377f63d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:16:51 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:44648
I20260811 17:16:51.043192 5608 tablet_service.cc:1514] Processing DeleteTablet for tablet 91dc3c3d685b45dc9dbbaab6b377f63d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:16:51 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:46072
I20260811 17:16:51.043766 6041 tablet_replica.cc:330] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240: stopping tablet replica
I20260811 17:16:51.043787 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 5535
I20260811 17:16:51.044167 6041 raft_consensus.cc:2229] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:16:51.045490 6039 tablet_replica.cc:330] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23: stopping tablet replica
I20260811 17:16:51.045852 6039 raft_consensus.cc:2229] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:16:51.046236 6039 raft_consensus.cc:2258] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:16:51.046665 6040 tablet_replica.cc:330] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee: stopping tablet replica
I20260811 17:16:51.046962 6040 raft_consensus.cc:2229] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:16:51.047399 6040 raft_consensus.cc:2258] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:16:51.049403 6039 ts_tablet_manager.cc:1905] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:16:51.049537 6040 ts_tablet_manager.cc:1905] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:16:51.052597 6040 ts_tablet_manager.cc:1918] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:16:51.052717 6040 log.cc:1196] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/91dc3c3d685b45dc9dbbaab6b377f63d
I20260811 17:16:51.053089 6040 ts_tablet_manager.cc:1939] T 91dc3c3d685b45dc9dbbaab6b377f63d P 8692e983abb54c31bced385540a68fee: Deleting consensus metadata
I20260811 17:16:51.053812 6039 ts_tablet_manager.cc:1918] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:16:51.054052 6039 log.cc:1196] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/91dc3c3d685b45dc9dbbaab6b377f63d
I20260811 17:16:51.054522 6039 ts_tablet_manager.cc:1939] T 91dc3c3d685b45dc9dbbaab6b377f63d P d0e1c0675ae649f88adbeb4f0936fa23: Deleting consensus metadata
I20260811 17:16:51.054539 5447 catalog_manager.cc:4841] TS 8692e983abb54c31bced385540a68fee (127.21.54.130:45421): tablet 91dc3c3d685b45dc9dbbaab6b377f63d (table test-table [id=214f099180c54e39b26e845e7371979f]) successfully deleted
I20260811 17:16:51.056432 5449 catalog_manager.cc:4841] TS d0e1c0675ae649f88adbeb4f0936fa23 (127.21.54.131:38839): tablet 91dc3c3d685b45dc9dbbaab6b377f63d (table test-table [id=214f099180c54e39b26e845e7371979f]) successfully deleted
W20260811 17:16:51.060415 5449 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.54.129:37467 (error 108)
I20260811 17:16:51.061448 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 5693
I20260811 17:16:51.061697 6010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:51.061341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:37467 (local address 127.0.0.1:46086)
0811 17:16:51.061489 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:51.061579 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.129:37467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 17:16:51.067834 5449 catalog_manager.cc:4584] TS da6b2e8a85c64c859170faef63149240 (127.21.54.129:37467): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 91dc3c3d685b45dc9dbbaab6b377f63d: Network error: Client connection negotiation failed: client connection to 127.21.54.129:37467: connect: Connection refused (error 111)
I20260811 17:16:51.073922 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 5850
I20260811 17:16:51.081781 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 5427
I20260811 17:16:51.092392 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:45313
--webserver_interface=127.21.54.190
--webserver_port=42893
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:45313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:16:51.212071 6044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:51.212337 6044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:51.212378 6044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:51.216276 6044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:16:51.216331 6044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:51.216351 6044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:51.216367 6044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:16:51.216382 6044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:16:51.221987 6044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:45313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:45313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=42893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:51.223049 6044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:51.224045 6044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:51.241256 6051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:51.241217 6054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:51.241245 6050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:51.241314 6044 server_base.cc:1034] running on GCE node
I20260811 17:16:51.242221 6044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:51.243450 6044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:51.244601 6044 hybrid_clock.cc:648] HybridClock initialized: now 1786468611244581 us; error 28 us; skew 500 ppm
Aug 11 17:16:51 dist-test-slave-fd7l krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468611, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:51.247699 6044 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:16:51.248791 6044 webserver.cc:460] Webserver started at http://127.21.54.190:42893/ using document root <none> and password file <none>
I20260811 17:16:51.249471 6044 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:51.249590 6044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:51.254189 6044 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:16:51.256979 6062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:51.258525 6044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:16:51.258713 6044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d"
format_stamp: "Formatted at 2026-08-11 17:16:49 on dist-test-slave-fd7l"
server_key: "d0941b49ecbca6bf43dada315016e43c"
server_key_iv: "2ab20b81920d9dd74c8cf22520ce03c6"
server_key_version: "encryptionkey@0"
I20260811 17:16:51.259418 6044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:51.282006 6044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:51.285574 6044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:51.285804 6044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:51.293792 6044 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:45313
I20260811 17:16:51.294133 6121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:45313 every 8 connection(s)
I20260811 17:16:51.295014 6044 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:16:51.299803 6122 sys_catalog.cc:263] Verifying existing consensus state
I20260811 17:16:51.300575 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 6044
I20260811 17:16:51.301048 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:37467
--local_ip_for_outbound_sockets=127.21.54.129
--tserver_master_addrs=127.21.54.190:45313
--webserver_port=39869
--webserver_interface=127.21.54.129
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:16:51.301901 6122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d: Bootstrap starting.
I20260811 17:16:51.320641 6122 log.cc:824] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d: Log is configured to *not* fsync() on all Append() calls
I20260811 17:16:51.330221 6122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:16:51.330921 6122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d: Bootstrap complete.
I20260811 17:16:51.335469 6122 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45313 } }
I20260811 17:16:51.335963 6122 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc29907ebf7c4c7a8f23ebbe05ebb96d, State: Initialized, Role: FOLLOWER
I20260811 17:16:51.336632 6122 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45313 } }
I20260811 17:16:51.336795 6122 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:16:51.336898 6122 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:16:51.337031 6122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 1 FOLLOWER]: Advancing to term 2
I20260811 17:16:51.339206 6122 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45313 } }
I20260811 17:16:51.339612 6122 leader_election.cc:302] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc29907ebf7c4c7a8f23ebbe05ebb96d; no voters:
I20260811 17:16:51.339998 6122 leader_election.cc:288] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 17:16:51.340106 6126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 2 FOLLOWER]: Leader election won for term 2
I20260811 17:16:51.340327 6126 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 2 LEADER]: Becoming Leader. State: Replica: dc29907ebf7c4c7a8f23ebbe05ebb96d, State: Running, Role: LEADER
I20260811 17:16:51.340677 6126 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45313 } }
I20260811 17:16:51.341431 6122 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:16:51.342298 6128 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc29907ebf7c4c7a8f23ebbe05ebb96d. Latest consensus state: current_term: 2 leader_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45313 } } }
I20260811 17:16:51.342471 6128 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [sys.catalog]: This master's current role is: LEADER
I20260811 17:16:51.342860 6127 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45313 } } }
I20260811 17:16:51.342964 6127 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [sys.catalog]: This master's current role is: LEADER
I20260811 17:16:51.343772 6135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:16:51.348462 6135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:16:51.349088 6135 catalog_manager.cc:1259] Loaded cluster ID: 8e7bb94620e0442fbd9f4fd61801b4a2
I20260811 17:16:51.349171 6135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:16:51.350484 6135 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 17:16:51.353284 6135 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 17:16:51.358150 6135 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:16:51.358974 6135 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d: Loaded TSK: 0
I20260811 17:16:51.359998 6135 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 17:16:51.439221 6124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:51.439548 6124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:51.439638 6124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:51.444267 6124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:51.444375 6124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:51.444468 6124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:16:51.450163 6124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:37467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=39869
--tserver_master_addrs=127.21.54.190:45313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:51.451280 6124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:51.452316 6124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:51.462464 6156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:51.463032 6124 server_base.cc:1034] running on GCE node
W20260811 17:16:51.463061 6158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:51.463335 6155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:51.463879 6124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:51.464942 6124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:51.466148 6124 hybrid_clock.cc:648] HybridClock initialized: now 1786468611466127 us; error 37 us; skew 500 ppm
Aug 11 17:16:51 dist-test-slave-fd7l krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468611, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:51.470386 6124 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:16:51.471539 6124 webserver.cc:460] Webserver started at http://127.21.54.129:39869/ using document root <none> and password file <none>
I20260811 17:16:51.472256 6124 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:51.472378 6124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:51.476369 6124 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:16:51.478770 6166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:51.480115 6124 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:16:51.480253 6124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "da6b2e8a85c64c859170faef63149240"
format_stamp: "Formatted at 2026-08-11 17:16:50 on dist-test-slave-fd7l"
server_key: "b6987468bdddf967cddea6347243c39f"
server_key_iv: "f6ca41454b63fa28ca5ed1d650aac463"
server_key_version: "encryptionkey@0"
I20260811 17:16:51.480906 6124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:51.502054 6124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:51.505678 6124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:51.505914 6124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:51.506665 6124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:16:51.507867 6175 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 17:16:51.512384 6124 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 17:16:51.512557 6124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 17:16:51.512666 6124 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 17:16:51.514422 6124 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 17:16:51.514468 6124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 17:16:51.514814 6175 tablet_bootstrap.cc:492] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240: Bootstrap starting.
I20260811 17:16:51.525134 6124 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:37467
I20260811 17:16:51.525197 6299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:37467 every 8 connection(s)
I20260811 17:16:51.526098 6124 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:16:51.529872 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 6124
I20260811 17:16:51.530324 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:45421
--local_ip_for_outbound_sockets=127.21.54.130
--tserver_master_addrs=127.21.54.190:45313
--webserver_port=40723
--webserver_interface=127.21.54.130
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:16:51.534974 6175 log.cc:824] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240: Log is configured to *not* fsync() on all Append() calls
Aug 11 17:16:51 dist-test-slave-fd7l krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468611, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:16:51.543562 6175 tablet_bootstrap.cc:492] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:16:51.544027 6175 tablet_bootstrap.cc:492] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240: Bootstrap complete.
I20260811 17:16:51.545486 6175 ts_tablet_manager.cc:1397] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240: Time spent bootstrapping tablet: real 0.031s user 0.019s sys 0.010s
I20260811 17:16:51.548231 6303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:51.528637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:44651 (local address 127.21.54.190:45313)
0811 17:16:51.529116 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:51.529137 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:51.529491 (+ 354us) server_negotiation.cc:407] Connection header received
0811 17:16:51.531130 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:51.531148 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:51.531567 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:51.531951 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:51.533710 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.535315 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.537447 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.538340 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.538555 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:51.541825 (+ 3270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:51.541833 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:51.541853 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:51.541898 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:51.544436 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:51.545227 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:51.545233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:51.545239 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:51.545396 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:51.545813 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:51.545817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:51.545819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:51.546298 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:51.546431 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:51.547078 (+ 647us) server_negotiation.cc:299] Negotiation successful
0811 17:16:51.547390 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":146,"threads_started":1}
I20260811 17:16:51.548255 6304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:51.528603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:44651)
0811 17:16:51.529289 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:51.529350 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:51.530603 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:51.532071 (+ 1468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:51.532091 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:51.532766 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:51.533486 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:51.533509 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.535522 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.535529 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:51.537263 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:51.537280 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.538528 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.538532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:51.538723 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:51.539484 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:51.539507 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:51.541563 (+ 2056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:51.544654 (+ 3091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:51.544663 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:51.544680 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:51.545058 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:51.545510 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:51.545513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:51.545515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:51.545692 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:51.546426 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:51.546433 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:51.546718 (+ 285us) client_negotiation.cc:769] Sending connection context
0811 17:16:51.546935 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 17:16:51.547289 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":168,"threads_started":1}
I20260811 17:16:51.548450 6175 raft_consensus.cc:348] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } }
I20260811 17:16:51.549033 6175 raft_consensus.cc:729] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da6b2e8a85c64c859170faef63149240, State: Initialized, Role: FOLLOWER
I20260811 17:16:51.549561 6175 consensus_queue.cc:260] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } }
I20260811 17:16:51.550552 6175 ts_tablet_manager.cc:1428] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20260811 17:16:51.551203 6301 heartbeater.cc:347] Connected to a master server at 127.21.54.190:45313
I20260811 17:16:51.551460 6301 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:51.551954 6301 heartbeater.cc:511] Master 127.21.54.190:45313 requested a full tablet report, sending...
I20260811 17:16:51.554615 6081 ts_manager.cc:194] Registered new tserver with Master: da6b2e8a85c64c859170faef63149240 (127.21.54.129:37467)
I20260811 17:16:51.558823 6081 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:44651
I20260811 17:16:51.561479 6301 heartbeater.cc:503] Master 127.21.54.190:45313 was elected leader, sending a full tablet report...
Aug 11 17:16:51 dist-test-slave-fd7l krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468611, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for kudu/127.21.54.129@KRBTEST.COM
I20260811 17:16:51.570046 6308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:51.558185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:37467 (local address 127.0.0.1:46096)
0811 17:16:51.558982 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:51.559031 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:51.559215 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:51.559798 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:51.559804 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:51.560286 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:51.560421 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:51.560430 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.561811 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.561815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:51.562661 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:51.562672 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.563439 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.563443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:51.563596 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:51.564240 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:51.564257 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:51.566174 (+ 1917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:51.568141 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:51.568152 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:51.568158 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:51.568401 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:51.568748 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:51.568752 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:51.568754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:51.568848 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:51.569338 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:51.569344 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:51.569483 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 17:16:51.569638 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 17:16:51.569856 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":128,"threads_started":1}
I20260811 17:16:51.570046 6309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:51.558236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46096 (local address 127.21.54.129:37467)
0811 17:16:51.558703 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:51.558716 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:51.559150 (+ 434us) server_negotiation.cc:407] Connection header received
0811 17:16:51.559385 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:51.559390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:51.559551 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:51.559728 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:51.560603 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.561676 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.562785 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.563315 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.563437 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:51.566333 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:51.566343 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:51.566353 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:51.566376 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:51.568006 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:51.568546 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:51.568554 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:51.568560 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:51.568644 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:51.568945 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:51.568949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:51.568951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:51.569212 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:51.569367 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:51.569725 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 17:16:51.569892 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":106,"threads_started":1}
I20260811 17:16:51.571189 6224 tablet_service.cc:1514] Processing DeleteTablet for tablet 91dc3c3d685b45dc9dbbaab6b377f63d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:16:51 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:46096
I20260811 17:16:51.572377 6310 tablet_replica.cc:330] stopping tablet replica
I20260811 17:16:51.572758 6310 raft_consensus.cc:2229] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:16:51.573029 6310 raft_consensus.cc:2258] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:16:51.574028 6310 ts_tablet_manager.cc:1905] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:16:51.577782 6310 ts_tablet_manager.cc:1918] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:16:51.577906 6310 log.cc:1196] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/wal/wals/91dc3c3d685b45dc9dbbaab6b377f63d
I20260811 17:16:51.578183 6310 ts_tablet_manager.cc:1939] T 91dc3c3d685b45dc9dbbaab6b377f63d P da6b2e8a85c64c859170faef63149240: Deleting consensus metadata
I20260811 17:16:51.580060 6065 catalog_manager.cc:4841] TS da6b2e8a85c64c859170faef63149240 (127.21.54.129:37467): tablet 91dc3c3d685b45dc9dbbaab6b377f63d (table test-table [id=214f099180c54e39b26e845e7371979f]) successfully deleted
W20260811 17:16:51.659008 6305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:51.659271 6305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:51.659313 6305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:51.663404 6305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:51.663463 6305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:51.663527 6305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:16:51.669010 6305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:45421
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=40723
--tserver_master_addrs=127.21.54.190:45313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:51.670153 6305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:51.671227 6305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:16:51.681465 6305 server_base.cc:1034] running on GCE node
W20260811 17:16:51.681398 6318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:51.681404 6315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:51.681404 6316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:51.682132 6305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:51.682902 6305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:51.684098 6305 hybrid_clock.cc:648] HybridClock initialized: now 1786468611684080 us; error 29 us; skew 500 ppm
Aug 11 17:16:51 dist-test-slave-fd7l krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468611, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:51.687328 6305 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:16:51.688316 6305 webserver.cc:460] Webserver started at http://127.21.54.130:40723/ using document root <none> and password file <none>
I20260811 17:16:51.689014 6305 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:51.689134 6305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:51.693219 6305 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:16:51.695390 6327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:51.696749 6305 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 17:16:51.696897 6305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "8692e983abb54c31bced385540a68fee"
format_stamp: "Formatted at 2026-08-11 17:16:50 on dist-test-slave-fd7l"
server_key: "b32d048c1a92ffc7c7eb2c07b47a031a"
server_key_iv: "cb5149d50e779e342b17080b44540df6"
server_key_version: "encryptionkey@0"
I20260811 17:16:51.697466 6305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:51.723809 6305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:51.727154 6305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:51.727377 6305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:51.728052 6305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:16:51.728957 6305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:16:51.729036 6305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:51.729111 6305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:16:51.729169 6305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:51.738607 6305 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:45421
I20260811 17:16:51.738626 6456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:45421 every 8 connection(s)
I20260811 17:16:51.739670 6305 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:16:51.749845 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 6305
I20260811 17:16:51.750422 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:38839
--local_ip_for_outbound_sockets=127.21.54.131
--tserver_master_addrs=127.21.54.190:45313
--webserver_port=40469
--webserver_interface=127.21.54.131
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 17:16:51 dist-test-slave-fd7l krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468611, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:16:51.755720 6303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:51.741790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:56099 (local address 127.21.54.190:45313)
0811 17:16:51.741934 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:51.741938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:51.742892 (+ 954us) server_negotiation.cc:407] Connection header received
0811 17:16:51.744562 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:51.744565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:51.744638 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:51.744760 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:51.746067 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.746645 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.747900 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.748171 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.748295 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:51.751469 (+ 3174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:51.751476 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:51.751482 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:51.751531 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:51.753220 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:51.753917 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:51.753919 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:51.753921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:51.754006 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:51.754434 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:51.754436 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:51.754437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:51.754634 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:51.754718 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:51.755397 (+ 679us) server_negotiation.cc:299] Negotiation successful
0811 17:16:51.755580 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 17:16:51.757069 6460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:51.742191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:56099)
0811 17:16:51.742711 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:51.742757 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:51.744291 (+ 1534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:51.744959 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:51.744966 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:51.745366 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:51.745892 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:51.745906 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.746784 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.746789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:51.747755 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:51.747765 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.748342 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.748347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:51.748611 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:51.749301 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:51.749319 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:51.751276 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:51.753357 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:51.753367 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:51.753386 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:51.753796 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:51.754120 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:51.754127 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:51.754131 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:51.754330 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:51.754748 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:51.754758 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:51.755126 (+ 368us) client_negotiation.cc:769] Sending connection context
0811 17:16:51.755359 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 17:16:51.755774 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":115,"threads_started":1}
I20260811 17:16:51.758543 6458 heartbeater.cc:347] Connected to a master server at 127.21.54.190:45313
I20260811 17:16:51.758826 6458 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:51.759351 6458 heartbeater.cc:511] Master 127.21.54.190:45313 requested a full tablet report, sending...
I20260811 17:16:51.760579 6081 ts_manager.cc:194] Registered new tserver with Master: 8692e983abb54c31bced385540a68fee (127.21.54.130:45421)
I20260811 17:16:51.761197 6081 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:56099
W20260811 17:16:51.869499 6461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:51.869812 6461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:51.869884 6461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:51.874075 6461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:51.874176 6461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:51.874272 6461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:16:51.880779 6461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:38839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=40469
--tserver_master_addrs=127.21.54.190:45313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:51.882004 6461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:51.883136 6461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:16:51.894304 6461 server_base.cc:1034] running on GCE node
W20260811 17:16:51.894228 6468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:51.894222 6467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:51.894248 6470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:51.895031 6461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:51.895704 6461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:51.896869 6461 hybrid_clock.cc:648] HybridClock initialized: now 1786468611896856 us; error 26 us; skew 500 ppm
Aug 11 17:16:51 dist-test-slave-fd7l krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468611, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:51.899652 6461 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:16:51.900547 6461 webserver.cc:460] Webserver started at http://127.21.54.131:40469/ using document root <none> and password file <none>
I20260811 17:16:51.901183 6461 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:51.901263 6461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:51.905311 6461 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:16:51.907284 6477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:51.908454 6461 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 17:16:51.908648 6461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "d0e1c0675ae649f88adbeb4f0936fa23"
format_stamp: "Formatted at 2026-08-11 17:16:50 on dist-test-slave-fd7l"
server_key: "b50e7419b2977babf80ee9dad1ce8731"
server_key_iv: "d1f7db4d4266afd6997bf46e55710e53"
server_key_version: "encryptionkey@0"
I20260811 17:16:51.909209 6461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:51.922348 6461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:51.925463 6461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:51.925680 6461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:51.926318 6461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:16:51.927197 6461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:16:51.927245 6461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:51.927316 6461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:16:51.927352 6461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:51.936293 6461 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:38839
I20260811 17:16:51.936324 6608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:38839 every 8 connection(s)
I20260811 17:16:51.937402 6461 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:16:51.939785 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 6461
Aug 11 17:16:51 dist-test-slave-fd7l krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468611, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:16:51.950425 6615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:51.941115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50248 (local address 127.21.54.190:45313)
0811 17:16:51.941385 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:51.941389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:51.941399 (+ 10us) server_negotiation.cc:407] Connection header received
0811 17:16:51.941438 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:51.941440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:51.941480 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:51.941590 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:51.942427 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.943130 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.944128 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.944390 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.944489 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:51.946115 (+ 1626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:51.946118 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:51.946120 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:51.946143 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:51.947745 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:51.948276 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:51.948279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:51.948281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:51.948368 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:51.949267 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:51.949270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:51.949272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:51.949573 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:51.949804 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:51.950063 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 17:16:51.950213 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"spinlock_wait_cycles":1280,"thread_start_us":88,"threads_started":1}
I20260811 17:16:51.953523 6303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:51.939598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:52923 (local address 127.21.54.190:45313)
0811 17:16:51.939761 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:51.939767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:51.940987 (+ 1220us) server_negotiation.cc:407] Connection header received
0811 17:16:51.942923 (+ 1936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:51.942927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:51.942975 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:51.943058 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:51.944431 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.944994 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.946308 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.946576 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.946703 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:51.949264 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:51.949268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:51.949271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:51.949306 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:51.950995 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:51.951817 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:51.951821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:51.951822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:51.951910 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:51.952352 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:51.952356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:51.952357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:51.952603 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:51.952704 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:51.953307 (+ 603us) server_negotiation.cc:299] Negotiation successful
0811 17:16:51.953405 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":64,"spinlock_wait_cycles":2432}
I20260811 17:16:51.954528 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:51.940016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:52923)
0811 17:16:51.940768 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:51.940840 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:51.942718 (+ 1878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:51.943238 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:51.943246 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:51.943753 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:51.944291 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:51.944305 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.945133 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.945139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:51.946152 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:51.946163 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.946748 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.946753 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:51.946960 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:51.947388 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:51.947405 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:51.949083 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:51.951129 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:51.951136 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:51.951154 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:51.951567 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:51.952019 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:51.952023 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:51.952025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:51.952240 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:51.952706 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:51.952711 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:51.952993 (+ 282us) client_negotiation.cc:769] Sending connection context
0811 17:16:51.953251 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 17:16:51.953601 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":147,"threads_started":1}
I20260811 17:16:51.955722 6609 heartbeater.cc:347] Connected to a master server at 127.21.54.190:45313
I20260811 17:16:51.956012 6609 heartbeater.cc:464] Registering TS with master...
I20260811 17:16:51.956624 6609 heartbeater.cc:511] Master 127.21.54.190:45313 requested a full tablet report, sending...
I20260811 17:16:51.957736 6081 ts_manager.cc:194] Registered new tserver with Master: d0e1c0675ae649f88adbeb4f0936fa23 (127.21.54.131:38839)
I20260811 17:16:51.958365 6081 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:52923
I20260811 17:16:51.965565 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:16:51.970567 6303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:51.966259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50254 (local address 127.21.54.190:45313)
0811 17:16:51.966460 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:51.966465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:51.966658 (+ 193us) server_negotiation.cc:407] Connection header received
0811 17:16:51.966873 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:51.966875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:51.966984 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:51.967119 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:16:51.967375 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.968118 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.969071 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.969299 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.969406 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:51.969747 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:16:51.970172 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:16:51.970320 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:51.970364 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 17:16:51.970418 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260811 17:16:51.971347 6081 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:50254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:16:51.975000 6081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:16:51.983423 6224 tablet_service.cc:1467] Processing CreateTablet for tablet 007363f9b3384029b77d284b4fef93b4 (DEFAULT_TABLE table=test-table [id=a531207eaace46c1a91fa2b41a207e41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:16:51.984190 6224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 007363f9b3384029b77d284b4fef93b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:16:51.987478 6175 tablet_bootstrap.cc:492] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240: Bootstrap starting.
I20260811 17:16:51.988409 6175 tablet_bootstrap.cc:654] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240: Neither blocks nor log segments found. Creating new log.
I20260811 17:16:51.988842 6308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:51.982298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:38839 (local address 127.0.0.1:44654)
0811 17:16:51.982428 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:51.982445 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:51.982531 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:51.983749 (+ 1218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:51.983753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:51.983771 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:16:51.983897 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:51.983905 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.985694 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.985697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:51.986893 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:51.986905 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.988143 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.988147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:51.988382 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:51.988459 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 17:16:51.988568 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 17:16:51.988666 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":137}
I20260811 17:16:51.989161 6621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:51.982735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:45421 (local address 127.0.0.1:35542)
0811 17:16:51.983066 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:51.983096 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:51.983190 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:51.983878 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:51.983881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:51.983900 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:16:51.984000 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:51.984007 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.985486 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.985509 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:51.986893 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:51.986905 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.988142 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.988146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:51.988405 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:51.988481 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 17:16:51.988608 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 17:16:51.988713 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":58,"thread_start_us":103,"threads_started":1}
I20260811 17:16:51.989207 6622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:51.982902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35542 (local address 127.21.54.130:45421)
0811 17:16:51.983401 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:51.983413 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:51.983432 (+ 19us) server_negotiation.cc:407] Connection header received
0811 17:16:51.983492 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:51.983496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:51.983682 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:51.983878 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:16:51.984137 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.985329 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.987046 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.987995 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.988129 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:51.988294 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:51.988920 (+ 626us) server_negotiation.cc:299] Negotiation successful
0811 17:16:51.989046 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":118,"threads_started":1}
I20260811 17:16:51.990242 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:51.982483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44654 (local address 127.21.54.131:38839)
0811 17:16:51.983168 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:51.983183 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:51.983210 (+ 27us) server_negotiation.cc:407] Connection header received
0811 17:16:51.983301 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:51.983307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:51.983539 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:51.984114 (+ 575us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:16:51.984282 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.985543 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.987046 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:51.987994 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:51.988375 (+ 381us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:51.988670 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:51.988728 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 17:16:51.988821 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":184,"threads_started":1}
I20260811 17:16:51.991276 6175 tablet_bootstrap.cc:492] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240: No bootstrap required, opened a new log
I20260811 17:16:51.991377 6175 ts_tablet_manager.cc:1397] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 17:16:51.991849 6175 raft_consensus.cc:348] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } }
I20260811 17:16:51.991982 6175 raft_consensus.cc:374] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:16:51.992061 6175 raft_consensus.cc:729] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da6b2e8a85c64c859170faef63149240, State: Initialized, Role: FOLLOWER
I20260811 17:16:51.992223 6175 consensus_queue.cc:260] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } }
I20260811 17:16:51.992609 6175 ts_tablet_manager.cc:1428] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 17:16:51.992741 6383 tablet_service.cc:1467] Processing CreateTablet for tablet 007363f9b3384029b77d284b4fef93b4 (DEFAULT_TABLE table=test-table [id=a531207eaace46c1a91fa2b41a207e41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:16:51.993269 6531 tablet_service.cc:1467] Processing CreateTablet for tablet 007363f9b3384029b77d284b4fef93b4 (DEFAULT_TABLE table=test-table [id=a531207eaace46c1a91fa2b41a207e41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:16:51.994031 6383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 007363f9b3384029b77d284b4fef93b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:16:51.994272 6531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 007363f9b3384029b77d284b4fef93b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:16:52.001188 6626 tablet_bootstrap.cc:492] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee: Bootstrap starting.
I20260811 17:16:52.001157 6625 tablet_bootstrap.cc:492] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23: Bootstrap starting.
I20260811 17:16:52.003552 6625 tablet_bootstrap.cc:654] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23: Neither blocks nor log segments found. Creating new log.
I20260811 17:16:52.003753 6626 tablet_bootstrap.cc:654] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee: Neither blocks nor log segments found. Creating new log.
I20260811 17:16:52.004715 6626 log.cc:824] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee: Log is configured to *not* fsync() on all Append() calls
I20260811 17:16:52.004980 6625 log.cc:824] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23: Log is configured to *not* fsync() on all Append() calls
I20260811 17:16:52.006826 6626 tablet_bootstrap.cc:492] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee: No bootstrap required, opened a new log
I20260811 17:16:52.006901 6625 tablet_bootstrap.cc:492] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23: No bootstrap required, opened a new log
I20260811 17:16:52.007085 6626 ts_tablet_manager.cc:1397] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 17:16:52.007133 6625 ts_tablet_manager.cc:1397] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 17:16:52.010458 6626 raft_consensus.cc:348] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } }
I20260811 17:16:52.010727 6626 raft_consensus.cc:374] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:16:52.010797 6626 raft_consensus.cc:729] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8692e983abb54c31bced385540a68fee, State: Initialized, Role: FOLLOWER
I20260811 17:16:52.010675 6625 raft_consensus.cc:348] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } }
I20260811 17:16:52.010982 6625 raft_consensus.cc:374] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:16:52.011067 6625 raft_consensus.cc:729] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e1c0675ae649f88adbeb4f0936fa23, State: Initialized, Role: FOLLOWER
I20260811 17:16:52.011320 6626 consensus_queue.cc:260] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } }
I20260811 17:16:52.011479 6625 consensus_queue.cc:260] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } }
I20260811 17:16:52.012096 6458 heartbeater.cc:503] Master 127.21.54.190:45313 was elected leader, sending a full tablet report...
I20260811 17:16:52.012331 6626 ts_tablet_manager.cc:1428] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260811 17:16:52.012516 6625 ts_tablet_manager.cc:1428] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 17:16:52.012890 6609 heartbeater.cc:503] Master 127.21.54.190:45313 was elected leader, sending a full tablet report...
I20260811 17:16:52.026569 6629 raft_consensus.cc:482] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:16:52.026764 6629 raft_consensus.cc:504] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } }
W20260811 17:16:52.027346 6302 tablet.cc:2369] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:16:52.028127 6629 leader_election.cc:288] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0e1c0675ae649f88adbeb4f0936fa23 (127.21.54.131:38839), da6b2e8a85c64c859170faef63149240 (127.21.54.129:37467)
I20260811 17:16:52.032541 6460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:52.028459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:38839 (local address 127.21.54.130:55647)
0811 17:16:52.028664 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:52.028683 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:52.028809 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:52.029208 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:52.029211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:52.029231 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:16:52.029365 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:52.029375 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:52.030102 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:52.030107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:52.031150 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:52.031161 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:52.032062 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:52.032066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:52.032193 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:52.032213 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 17:16:52.032305 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 17:16:52.032381 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 17:16:52.032555 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:52.028593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:55647 (local address 127.21.54.131:38839)
0811 17:16:52.028784 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:52.028790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:52.028806 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:16:52.028952 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:52.028955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:52.029009 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:52.029083 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:16:52.029471 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:52.029976 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:52.031302 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:52.031937 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:52.032071 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:52.032129 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:52.032326 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 17:16:52.032405 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 17:16:52.032863 6309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:52.028943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:51419 (local address 127.21.54.129:37467)
0811 17:16:52.029083 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:52.029088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:52.029315 (+ 227us) server_negotiation.cc:407] Connection header received
0811 17:16:52.029407 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:52.029409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:52.029460 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:52.029640 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:16:52.029913 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:52.030470 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:52.031864 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:52.032311 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:52.032451 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:52.032651 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:52.032690 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 17:16:52.032757 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 17:16:52.032860 6632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:52.028804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:37467 (local address 127.21.54.130:51419)
0811 17:16:52.029220 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:52.029232 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:52.029325 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:52.029651 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:52.029654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:52.029671 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:16:52.029784 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:52.029791 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:52.030606 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:52.030609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:52.031742 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:52.031752 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:52.032441 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:52.032445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:52.032565 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:52.032587 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 17:16:52.032653 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 17:16:52.032724 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":147,"threads_started":1}
I20260811 17:16:52.033488 6555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "007363f9b3384029b77d284b4fef93b4" candidate_uuid: "8692e983abb54c31bced385540a68fee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" is_pre_election: true
I20260811 17:16:52.033628 6247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "007363f9b3384029b77d284b4fef93b4" candidate_uuid: "8692e983abb54c31bced385540a68fee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da6b2e8a85c64c859170faef63149240" is_pre_election: true
I20260811 17:16:52.033841 6555 raft_consensus.cc:2454] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8692e983abb54c31bced385540a68fee in term 0.
I20260811 17:16:52.033964 6247 raft_consensus.cc:2454] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8692e983abb54c31bced385540a68fee in term 0.
I20260811 17:16:52.034370 6330 leader_election.cc:302] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8692e983abb54c31bced385540a68fee, d0e1c0675ae649f88adbeb4f0936fa23; no voters:
I20260811 17:16:52.034641 6629 raft_consensus.cc:2781] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:16:52.034747 6629 raft_consensus.cc:482] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:16:52.034813 6629 raft_consensus.cc:3037] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:16:52.035890 6629 raft_consensus.cc:504] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } }
I20260811 17:16:52.036254 6629 leader_election.cc:288] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [CANDIDATE]: Term 1 election: Requested vote from peers d0e1c0675ae649f88adbeb4f0936fa23 (127.21.54.131:38839), da6b2e8a85c64c859170faef63149240 (127.21.54.129:37467)
I20260811 17:16:52.036667 6555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "007363f9b3384029b77d284b4fef93b4" candidate_uuid: "8692e983abb54c31bced385540a68fee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0e1c0675ae649f88adbeb4f0936fa23"
I20260811 17:16:52.036818 6247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "007363f9b3384029b77d284b4fef93b4" candidate_uuid: "8692e983abb54c31bced385540a68fee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da6b2e8a85c64c859170faef63149240"
I20260811 17:16:52.036800 6555 raft_consensus.cc:3037] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:16:52.036971 6247 raft_consensus.cc:3037] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:16:52.037981 6555 raft_consensus.cc:2454] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8692e983abb54c31bced385540a68fee in term 1.
I20260811 17:16:52.038015 6247 raft_consensus.cc:2454] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8692e983abb54c31bced385540a68fee in term 1.
I20260811 17:16:52.038357 6330 leader_election.cc:302] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8692e983abb54c31bced385540a68fee, da6b2e8a85c64c859170faef63149240; no voters:
I20260811 17:16:52.038561 6629 raft_consensus.cc:2781] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:16:52.038823 6629 raft_consensus.cc:686] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [term 1 LEADER]: Becoming Leader. State: Replica: 8692e983abb54c31bced385540a68fee, State: Running, Role: LEADER
I20260811 17:16:52.039172 6629 consensus_queue.cc:237] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } }
I20260811 17:16:52.042529 6081 catalog_manager.cc:5495] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee reported cstate change: term changed from 0 to 1, leader changed from <none> to 8692e983abb54c31bced385540a68fee (127.21.54.130). New cstate: current_term: 1 leader_uuid: "8692e983abb54c31bced385540a68fee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } health_report { overall_health: UNKNOWN } } }
I20260811 17:16:52.062834 6622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:52.059075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35554 (local address 127.21.54.130:45421)
0811 17:16:52.059220 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:52.059225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:52.059239 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:16:52.059406 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:52.059409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:52.059525 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:52.059647 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:16:52.059896 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:52.060397 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:52.061270 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:52.061498 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:52.061607 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:52.061922 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:16:52.062384 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:16:52.062566 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:52.062601 (+ 35us) server_negotiation.cc:299] Negotiation successful
0811 17:16:52.062658 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 17:16:52.068897 6247 raft_consensus.cc:1261] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Refusing update from remote peer 8692e983abb54c31bced385540a68fee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:16:52.068939 6555 raft_consensus.cc:1261] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23 [term 1 FOLLOWER]: Refusing update from remote peer 8692e983abb54c31bced385540a68fee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:16:52.069631 6629 consensus_queue.cc:1035] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [LEADER]: Connected to new peer: Peer: permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:16:52.069860 6634 consensus_queue.cc:1035] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:16:52.075110 6636 mvcc.cc:205] Tried to move back new op lower bound from 7317375435026599936 to 7317375434915205120. Current Snapshot: MvccSnapshot[applied={T|T < 7317375435026599936}]
I20260811 17:16:52.077394 6637 mvcc.cc:205] Tried to move back new op lower bound from 7317375435026599936 to 7317375434915205120. Current Snapshot: MvccSnapshot[applied={T|T < 7317375435026599936}]
I20260811 17:16:52.079514 6638 mvcc.cc:205] Tried to move back new op lower bound from 7317375435026599936 to 7317375434915205120. Current Snapshot: MvccSnapshot[applied={T|T < 7317375435026599936}]
I20260811 17:16:52.084151 6080 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:50254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:16:52.084304 6080 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:50254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:16:52.086956 6080 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d: Sending DeleteTablet for 3 replicas of tablet 007363f9b3384029b77d284b4fef93b4
I20260811 17:16:52.087693 6224 tablet_service.cc:1514] Processing DeleteTablet for tablet 007363f9b3384029b77d284b4fef93b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:16:52 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:46096
I20260811 17:16:52.087693 6531 tablet_service.cc:1514] Processing DeleteTablet for tablet 007363f9b3384029b77d284b4fef93b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:16:52 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:44654
I20260811 17:16:52.088027 6647 tablet_replica.cc:330] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240: stopping tablet replica
I20260811 17:16:52.088160 6647 raft_consensus.cc:2229] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:16:52.088203 6383 tablet_service.cc:1514] Processing DeleteTablet for tablet 007363f9b3384029b77d284b4fef93b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:16:52 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:35542
I20260811 17:16:52.088311 6647 raft_consensus.cc:2258] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:16:52.088385 6648 tablet_replica.cc:330] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23: stopping tablet replica
I20260811 17:16:52.088775 6648 raft_consensus.cc:2229] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:16:52.089176 6648 raft_consensus.cc:2258] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:16:52.089190 6647 ts_tablet_manager.cc:1905] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:16:52.089229 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 6124
I20260811 17:16:52.093962 6648 ts_tablet_manager.cc:1905] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:16:52.094323 6649 tablet_replica.cc:330] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee: stopping tablet replica
I20260811 17:16:52.094681 6649 raft_consensus.cc:2229] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:16:52.095101 6649 raft_consensus.cc:2258] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:16:52.096619 6649 ts_tablet_manager.cc:1905] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:16:52.097347 6648 ts_tablet_manager.cc:1918] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:16:52.097519 6648 log.cc:1196] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/007363f9b3384029b77d284b4fef93b4
I20260811 17:16:52.097971 6648 ts_tablet_manager.cc:1939] T 007363f9b3384029b77d284b4fef93b4 P d0e1c0675ae649f88adbeb4f0936fa23: Deleting consensus metadata
I20260811 17:16:52.099422 6065 catalog_manager.cc:4841] TS d0e1c0675ae649f88adbeb4f0936fa23 (127.21.54.131:38839): tablet 007363f9b3384029b77d284b4fef93b4 (table test-table [id=a531207eaace46c1a91fa2b41a207e41]) successfully deleted
I20260811 17:16:52.100059 6649 ts_tablet_manager.cc:1918] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:16:52.100172 6649 log.cc:1196] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/007363f9b3384029b77d284b4fef93b4
I20260811 17:16:52.100541 6649 ts_tablet_manager.cc:1939] T 007363f9b3384029b77d284b4fef93b4 P 8692e983abb54c31bced385540a68fee: Deleting consensus metadata
I20260811 17:16:52.101758 6063 catalog_manager.cc:4841] TS 8692e983abb54c31bced385540a68fee (127.21.54.130:45421): tablet 007363f9b3384029b77d284b4fef93b4 (table test-table [id=a531207eaace46c1a91fa2b41a207e41]) successfully deleted
W20260811 17:16:52.102627 6065 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.54.129:37467 (error 108)
I20260811 17:16:52.102933 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 6305
I20260811 17:16:52.111200 6621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:52.110690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:37467 (local address 127.0.0.1:46106)
0811 17:16:52.110948 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:52.111066 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.129:37467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260811 17:16:52.111509 6065 catalog_manager.cc:4584] TS da6b2e8a85c64c859170faef63149240 (127.21.54.129:37467): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 007363f9b3384029b77d284b4fef93b4: Network error: Client connection negotiation failed: client connection to 127.21.54.129:37467: connect: Connection refused (error 111)
I20260811 17:16:52.117547 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 6461
I20260811 17:16:52.126202 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 6044
I20260811 17:16:52.137784 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:45313
--webserver_interface=127.21.54.190
--webserver_port=42893
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:45313
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:16:52.255127 6652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:52.255404 6652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:52.255491 6652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:52.259573 6652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:16:52.259639 6652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:52.259660 6652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:52.259675 6652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:16:52.259691 6652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:16:52.265101 6652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:45313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:45313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=42893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:52.266238 6652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:52.267334 6652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:52.276331 6659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:52.276355 6658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:52.276341 6663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:52.276574 6652 server_base.cc:1034] running on GCE node
I20260811 17:16:52.277271 6652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:52.278330 6652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:52.279500 6652 hybrid_clock.cc:648] HybridClock initialized: now 1786468612279463 us; error 48 us; skew 500 ppm
Aug 11 17:16:52 dist-test-slave-fd7l krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468612, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:52.282390 6652 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:16:52.283304 6652 webserver.cc:460] Webserver started at http://127.21.54.190:42893/ using document root <none> and password file <none>
I20260811 17:16:52.283880 6652 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:52.283944 6652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:52.288242 6652 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.002s
I20260811 17:16:52.290669 6671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:52.292074 6652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:16:52.292179 6652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d"
format_stamp: "Formatted at 2026-08-11 17:16:49 on dist-test-slave-fd7l"
server_key: "d0941b49ecbca6bf43dada315016e43c"
server_key_iv: "2ab20b81920d9dd74c8cf22520ce03c6"
server_key_version: "encryptionkey@0"
I20260811 17:16:52.292860 6652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:52.305382 6652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:52.308326 6652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:52.308568 6652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:52.315992 6652 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:45313
I20260811 17:16:52.316308 6730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:45313 every 8 connection(s)
I20260811 17:16:52.317233 6652 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:16:52.321825 6731 sys_catalog.cc:263] Verifying existing consensus state
I20260811 17:16:52.323544 6731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d: Bootstrap starting.
I20260811 17:16:52.325155 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 6652
I20260811 17:16:52.325655 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:37467
--local_ip_for_outbound_sockets=127.21.54.129
--tserver_master_addrs=127.21.54.190:45313
--webserver_port=39869
--webserver_interface=127.21.54.129
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:16:52.341631 6731 log.cc:824] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d: Log is configured to *not* fsync() on all Append() calls
I20260811 17:16:52.353467 6731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:16:52.353888 6731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d: Bootstrap complete.
I20260811 17:16:52.358103 6731 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45313 } }
I20260811 17:16:52.358522 6731 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc29907ebf7c4c7a8f23ebbe05ebb96d, State: Initialized, Role: FOLLOWER
I20260811 17:16:52.359028 6731 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45313 } }
I20260811 17:16:52.359184 6731 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:16:52.359277 6731 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:16:52.359402 6731 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 2 FOLLOWER]: Advancing to term 3
I20260811 17:16:52.361431 6731 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45313 } }
I20260811 17:16:52.361788 6731 leader_election.cc:302] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc29907ebf7c4c7a8f23ebbe05ebb96d; no voters:
I20260811 17:16:52.362179 6731 leader_election.cc:288] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 17:16:52.362267 6736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 3 FOLLOWER]: Leader election won for term 3
I20260811 17:16:52.362471 6736 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [term 3 LEADER]: Becoming Leader. State: Replica: dc29907ebf7c4c7a8f23ebbe05ebb96d, State: Running, Role: LEADER
I20260811 17:16:52.362772 6736 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45313 } }
I20260811 17:16:52.363473 6731 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:16:52.363754 6736 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc29907ebf7c4c7a8f23ebbe05ebb96d. Latest consensus state: current_term: 3 leader_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45313 } } }
I20260811 17:16:52.363881 6736 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [sys.catalog]: This master's current role is: LEADER
I20260811 17:16:52.364262 6740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:16:52.364354 6737 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc29907ebf7c4c7a8f23ebbe05ebb96d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45313 } } }
I20260811 17:16:52.364578 6737 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc29907ebf7c4c7a8f23ebbe05ebb96d [sys.catalog]: This master's current role is: LEADER
I20260811 17:16:52.369797 6740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:16:52.370560 6740 catalog_manager.cc:1259] Loaded cluster ID: 8e7bb94620e0442fbd9f4fd61801b4a2
I20260811 17:16:52.370646 6740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 17:16:52.371641 6740 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8d766df9ad google::LogMessage::Fail() at ??:0
@ 0x7f8d766e288a google::LogMessage::SendToLog() at ??:0
@ 0x7f8d766df4de google::LogMessage::Flush() at ??:0
@ 0x7f8d766e0a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8d8101efdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f8d8101fd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8d8101c579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f8d81060640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f8d789313d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8d78931c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f8d789333f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f8d78922512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f8d79f666db start_thread at ??:0
@ 0x7f8d751dd61f clone at ??:0
W20260811 17:16:52.464448 6734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:52.464839 6734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:52.464891 6734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:52.469244 6734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:52.469331 6734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:52.469416 6734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:16:52.475010 6734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:37467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=39869
--tserver_master_addrs=127.21.54.190:45313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:52.476202 6734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:52.477363 6734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:52.487411 6780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:52.487411 6777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:52.487600 6778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:52.487798 6734 server_base.cc:1034] running on GCE node
I20260811 17:16:52.488556 6734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:52.489360 6734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:52.490581 6734 hybrid_clock.cc:648] HybridClock initialized: now 1786468612490533 us; error 49 us; skew 500 ppm
Aug 11 17:16:52 dist-test-slave-fd7l krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468612, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:52.493530 6734 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:16:52.494513 6734 webserver.cc:460] Webserver started at http://127.21.54.129:39869/ using document root <none> and password file <none>
I20260811 17:16:52.495102 6734 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:52.495165 6734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:52.499294 6734 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:16:52.501760 6788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:52.503232 6734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:16:52.503377 6734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "da6b2e8a85c64c859170faef63149240"
format_stamp: "Formatted at 2026-08-11 17:16:50 on dist-test-slave-fd7l"
server_key: "b6987468bdddf967cddea6347243c39f"
server_key_iv: "f6ca41454b63fa28ca5ed1d650aac463"
server_key_version: "encryptionkey@0"
I20260811 17:16:52.504000 6734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:52.515357 6734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:52.519043 6734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:52.519271 6734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:52.519963 6734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:16:52.521284 6799 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 17:16:52.527248 6734 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 17:16:52.527351 6734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260811 17:16:52.527441 6734 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 17:16:52.529474 6734 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 17:16:52.529557 6734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260811 17:16:52.529875 6799 tablet_bootstrap.cc:492] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240: Bootstrap starting.
I20260811 17:16:52.537110 6799 log.cc:824] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240: Log is configured to *not* fsync() on all Append() calls
I20260811 17:16:52.540881 6734 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:37467
I20260811 17:16:52.540911 6917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:37467 every 8 connection(s)
I20260811 17:16:52.542017 6734 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:16:52.544057 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 6734
I20260811 17:16:52.544575 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:45421
--local_ip_for_outbound_sockets=127.21.54.130
--tserver_master_addrs=127.21.54.190:45313
--webserver_port=40723
--webserver_interface=127.21.54.130
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:16:52.547186 6799 tablet_bootstrap.cc:492] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:16:52.547672 6799 tablet_bootstrap.cc:492] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240: Bootstrap complete.
I20260811 17:16:52.548090 6799 ts_tablet_manager.cc:1397] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240: Time spent bootstrapping tablet: real 0.018s user 0.014s sys 0.000s
I20260811 17:16:52.550757 6799 raft_consensus.cc:348] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } }
I20260811 17:16:52.551254 6799 raft_consensus.cc:729] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da6b2e8a85c64c859170faef63149240, State: Initialized, Role: FOLLOWER
I20260811 17:16:52.551738 6799 consensus_queue.cc:260] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } }
I20260811 17:16:52.552760 6799 ts_tablet_manager.cc:1428] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
W20260811 17:16:52.623907 6921 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:16:52.544245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:55031)
0811 17:16:52.545804 (+ 1559us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:52.545870 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:52.547003 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:52.622857 (+ 75854us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1288,"thread_start_us":179,"threads_started":1}
W20260811 17:16:52.624624 6919 heartbeater.cc:644] Failed to heartbeat to 127.21.54.190:45313 (0 consecutive failures): Network error: Failed to ping master at 127.21.54.190:45313: Client connection negotiation failed: client connection to 127.21.54.190:45313: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 17:16:52.625506 6921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:52.625175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:53259)
0811 17:16:52.625331 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:52.625424 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 17:16:52.626291 6921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:52.626032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:53075)
0811 17:16:52.626156 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:52.626217 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 17:16:52.626621 6919 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 17:16:52.682848 6922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:52.683154 6922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:52.683202 6922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:52.687420 6922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:52.687490 6922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:52.687601 6922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:16:52.693019 6922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:45421
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=40723
--tserver_master_addrs=127.21.54.190:45313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:52.694157 6922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:52.695303 6922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:52.705201 6928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:52.705242 6922 server_base.cc:1034] running on GCE node
W20260811 17:16:52.705211 6929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:52.705202 6935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:52.705960 6922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:52.706611 6922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:52.707787 6922 hybrid_clock.cc:648] HybridClock initialized: now 1786468612707773 us; error 25 us; skew 500 ppm
Aug 11 17:16:52 dist-test-slave-fd7l krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468612, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:52.710813 6922 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:16:52.711745 6922 webserver.cc:460] Webserver started at http://127.21.54.130:40723/ using document root <none> and password file <none>
I20260811 17:16:52.712293 6922 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:52.712355 6922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:52.716382 6922 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:16:52.718688 6943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:52.720104 6922 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:16:52.720273 6922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "8692e983abb54c31bced385540a68fee"
format_stamp: "Formatted at 2026-08-11 17:16:50 on dist-test-slave-fd7l"
server_key: "b32d048c1a92ffc7c7eb2c07b47a031a"
server_key_iv: "cb5149d50e779e342b17080b44540df6"
server_key_version: "encryptionkey@0"
I20260811 17:16:52.720983 6922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:52.738230 6922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:52.741703 6922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:52.741885 6922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:52.742533 6922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:16:52.743398 6922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:16:52.743453 6922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:52.743498 6922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:16:52.743525 6922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:52.753189 6922 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:45421
I20260811 17:16:52.753266 7073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:45421 every 8 connection(s)
I20260811 17:16:52.754371 6922 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:16:52.758649 7077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:52.756739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:50199)
0811 17:16:52.757232 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:52.757348 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":129,"threads_started":1}
I20260811 17:16:52.759037 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 6922
W20260811 17:16:52.759276 7074 heartbeater.cc:644] Failed to heartbeat to 127.21.54.190:45313 (0 consecutive failures): Network error: Failed to ping master at 127.21.54.190:45313: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
I20260811 17:16:52.759582 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:38839
--local_ip_for_outbound_sockets=127.21.54.131
--tserver_master_addrs=127.21.54.190:45313
--webserver_port=40469
--webserver_interface=127.21.54.131
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:34189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:16:52.760006 7077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:52.759777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:58737)
0811 17:16:52.759893 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:52.759951 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 17:16:52.760754 7077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:52.760503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:54067)
0811 17:16:52.760663 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:52.760705 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 17:16:52.760991 7074 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 17:16:52.876958 7078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:16:52.877250 7078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:16:52.877296 7078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:16:52.881444 7078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:16:52.881510 7078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:16:52.881573 7078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:16:52.887426 7078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:34189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:38839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=40469
--tserver_master_addrs=127.21.54.190:45313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:16:52.888674 7078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:16:52.889773 7078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:16:52.900323 7088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:52.900424 7086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:16:52.900324 7084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:16:52.900650 7078 server_base.cc:1034] running on GCE node
I20260811 17:16:52.901178 7078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:16:52.902242 7078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:16:52.903466 7078 hybrid_clock.cc:648] HybridClock initialized: now 1786468612903431 us; error 49 us; skew 500 ppm
Aug 11 17:16:52 dist-test-slave-fd7l krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468612, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:16:52.906418 7078 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:16:52.907521 7078 webserver.cc:460] Webserver started at http://127.21.54.131:40469/ using document root <none> and password file <none>
I20260811 17:16:52.908134 7078 fs_manager.cc:359] Metadata directory not provided
I20260811 17:16:52.908206 7078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:16:52.912312 7078 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:16:52.914649 7095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:52.916280 7078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:16:52.916389 7078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "d0e1c0675ae649f88adbeb4f0936fa23"
format_stamp: "Formatted at 2026-08-11 17:16:50 on dist-test-slave-fd7l"
server_key: "b50e7419b2977babf80ee9dad1ce8731"
server_key_iv: "d1f7db4d4266afd6997bf46e55710e53"
server_key_version: "encryptionkey@0"
I20260811 17:16:52.917065 7078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:16:52.939033 7078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:16:52.942375 7078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:16:52.942610 7078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:16:52.943274 7078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:16:52.944187 7078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:16:52.944235 7078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:52.944307 7078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:16:52.944339 7078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:16:52.953636 7078 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:38839
I20260811 17:16:52.953660 7221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:38839 every 8 connection(s)
I20260811 17:16:52.954859 7078 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:16:52.958133 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 7078
I20260811 17:16:52.959416 7224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:52.957315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:49737)
0811 17:16:52.957868 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:52.958003 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":127,"threads_started":1}
W20260811 17:16:52.959926 7222 heartbeater.cc:644] Failed to heartbeat to 127.21.54.190:45313 (0 consecutive failures): Network error: Failed to ping master at 127.21.54.190:45313: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
I20260811 17:16:52.960685 7224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:52.960336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:40757)
0811 17:16:52.960476 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:52.960619 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 17:16:52.961408 7224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:52.961208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:58215)
0811 17:16:52.961311 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:52.961357 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 17:16:52.961659 7222 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 17:16:53.627801 7226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:53.627291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:49321)
0811 17:16:53.627634 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:53.627699 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":114,"threads_started":1}
I20260811 17:16:53.762097 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:53.761614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:41417)
0811 17:16:53.761916 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:53.761983 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":75,"threads_started":1}
I20260811 17:16:53.962921 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:53.962364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:51261)
0811 17:16:53.962724 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:53.962796 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":84,"threads_started":1}
I20260811 17:16:54.073463 7229 raft_consensus.cc:482] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:16:54.073755 7229 raft_consensus.cc:504] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } }
I20260811 17:16:54.074980 7229 leader_election.cc:288] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d0e1c0675ae649f88adbeb4f0936fa23 (127.21.54.131:38839), 8692e983abb54c31bced385540a68fee (127.21.54.130:45421)
Aug 11 17:16:54 dist-test-slave-fd7l krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468612, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.131@KRBTEST.COM
Aug 11 17:16:54 dist-test-slave-fd7l krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468612, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.130@KRBTEST.COM
I20260811 17:16:54.098690 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:54.075468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:52241 (local address 127.21.54.130:45421)
0811 17:16:54.075865 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:54.075903 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:54.075948 (+ 45us) server_negotiation.cc:407] Connection header received
0811 17:16:54.076651 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:54.076658 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:54.077005 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:54.077332 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:54.078641 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:54.079624 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:54.081200 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:54.081670 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:54.081856 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:54.093762 (+ 11906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:54.093768 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:54.093781 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:54.093816 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:54.095862 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:54.096455 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:54.096461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:54.096465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:54.096667 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:54.097050 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:54.097054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:54.097056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:54.097509 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:54.097666 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:54.098241 (+ 575us) server_negotiation.cc:299] Negotiation successful
0811 17:16:54.098517 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":66,"threads_started":1}
I20260811 17:16:54.098670 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:54.075288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:45421 (local address 127.21.54.129:52241)
0811 17:16:54.075698 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:54.075715 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:54.075826 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:54.077678 (+ 1852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:54.077681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:54.077920 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:54.078456 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:54.078472 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:54.079801 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:54.079804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:54.081067 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:54.081078 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:54.081803 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:54.081807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:54.082023 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:54.091084 (+ 9061us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:54.091131 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:54.093515 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:54.095999 (+ 2484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:54.096004 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:54.096006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:54.096321 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:54.096771 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:54.096774 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:54.096776 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:54.096947 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:54.097641 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:54.097647 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:54.097893 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 17:16:54.098114 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 17:16:54.098465 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":127,"spinlock_wait_cycles":84736,"thread_start_us":83,"threads_started":1}
I20260811 17:16:54.098773 7234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:54.075397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:54541 (local address 127.21.54.131:38839)
0811 17:16:54.075819 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 17:16:54.075842 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 17:16:54.075879 (+ 37us) server_negotiation.cc:407] Connection header received
0811 17:16:54.076642 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:16:54.076649 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:16:54.077012 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:16:54.077290 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:16:54.078672 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:54.079627 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:54.081196 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:16:54.081670 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:16:54.081857 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:54.093713 (+ 11856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:16:54.093721 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:16:54.093737 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:16:54.093774 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:16:54.095722 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:54.096453 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:54.096459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:54.096464 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:54.096695 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:16:54.097073 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:16:54.097076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:16:54.097078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:16:54.097508 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:16:54.097659 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 17:16:54.098262 (+ 603us) server_negotiation.cc:299] Negotiation successful
0811 17:16:54.098582 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":129,"threads_started":1}
I20260811 17:16:54.098785 7226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:54.075184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:38839 (local address 127.21.54.129:54541)
0811 17:16:54.075442 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:54.075462 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 17:16:54.075580 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:16:54.077395 (+ 1815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:16:54.077402 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:16:54.077920 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:16:54.078456 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:54.078472 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:54.079803 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:54.079805 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:54.081063 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:16:54.081073 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:16:54.081804 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:16:54.081807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:16:54.082022 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:16:54.091084 (+ 9062us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:16:54.091118 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:16:54.093355 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:16:54.095889 (+ 2534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:54.095896 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:54.095911 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:54.096313 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:54.096800 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:16:54.096803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:16:54.096805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:16:54.096957 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:16:54.097639 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:16:54.097646 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:16:54.097893 (+ 247us) client_negotiation.cc:769] Sending connection context
0811 17:16:54.098112 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 17:16:54.098575 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":297,"spinlock_wait_cycles":32640}
W20260811 17:16:54.101440 6792 leader_election.cc:341] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d0e1c0675ae649f88adbeb4f0936fa23 (127.21.54.131:38839): Not found: Tablet not found: 007363f9b3384029b77d284b4fef93b4
W20260811 17:16:54.101619 6789 leader_election.cc:341] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8692e983abb54c31bced385540a68fee (127.21.54.130:45421): Not found: Tablet not found: 007363f9b3384029b77d284b4fef93b4
I20260811 17:16:54.101724 6789 leader_election.cc:302] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da6b2e8a85c64c859170faef63149240; no voters: 8692e983abb54c31bced385540a68fee, d0e1c0675ae649f88adbeb4f0936fa23
I20260811 17:16:54.102015 7229 raft_consensus.cc:2726] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:16:54.629252 7237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:54.628794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:58989)
0811 17:16:54.629065 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:54.629153 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":78,"threads_started":1}
I20260811 17:16:54.763446 7238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:54.762966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:54191)
0811 17:16:54.763285 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:54.763345 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":131,"threads_started":1}
I20260811 17:16:54.964478 7239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:54.963914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:40279)
0811 17:16:54.964252 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:54.964322 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":95,"threads_started":1}
I20260811 17:16:55.623320 7240 raft_consensus.cc:482] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:16:55.623526 7240 raft_consensus.cc:504] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } }
I20260811 17:16:55.624006 7240 leader_election.cc:288] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d0e1c0675ae649f88adbeb4f0936fa23 (127.21.54.131:38839), 8692e983abb54c31bced385540a68fee (127.21.54.130:45421)
W20260811 17:16:55.624886 6792 leader_election.cc:341] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d0e1c0675ae649f88adbeb4f0936fa23 (127.21.54.131:38839): Not found: Tablet not found: 007363f9b3384029b77d284b4fef93b4
W20260811 17:16:55.625082 6789 leader_election.cc:341] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8692e983abb54c31bced385540a68fee (127.21.54.130:45421): Not found: Tablet not found: 007363f9b3384029b77d284b4fef93b4
I20260811 17:16:55.625197 6789 leader_election.cc:302] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da6b2e8a85c64c859170faef63149240; no voters: 8692e983abb54c31bced385540a68fee, d0e1c0675ae649f88adbeb4f0936fa23
I20260811 17:16:55.625363 7240 raft_consensus.cc:2726] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:16:55.630534 7241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:55.630104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:46977)
0811 17:16:55.630382 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:55.630444 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":118,"threads_started":1}
I20260811 17:16:55.764966 7243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:55.764419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:57021)
0811 17:16:55.764798 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:55.764862 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":171,"threads_started":1}
I20260811 17:16:55.966158 7244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:55.965572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:42063)
0811 17:16:55.965903 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:55.965984 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":142,"threads_started":1}
I20260811 17:16:56.632037 7246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:56.631500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:45181)
0811 17:16:56.631862 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:56.631932 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":116,"threads_started":1}
I20260811 17:16:56.766328 7247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:56.765828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:45339)
0811 17:16:56.766138 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:56.766217 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":82,"threads_started":1}
I20260811 17:16:56.967794 7248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:56.967299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:57937)
0811 17:16:56.967598 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:56.967684 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":159,"threads_started":1}
I20260811 17:16:57.633730 7249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:57.633210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:33819)
0811 17:16:57.633546 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:57.633629 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":83,"threads_started":1}
I20260811 17:16:57.767925 7250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:57.767347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:48883)
0811 17:16:57.767683 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:57.767787 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":116,"threads_started":1}
I20260811 17:16:57.969317 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:57.968816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:45589)
0811 17:16:57.969158 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:57.969217 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":125,"threads_started":1}
I20260811 17:16:58.166532 7252 raft_consensus.cc:482] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:16:58.166687 7252 raft_consensus.cc:504] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } }
I20260811 17:16:58.167254 7252 leader_election.cc:288] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d0e1c0675ae649f88adbeb4f0936fa23 (127.21.54.131:38839), 8692e983abb54c31bced385540a68fee (127.21.54.130:45421)
W20260811 17:16:58.168264 6792 leader_election.cc:341] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d0e1c0675ae649f88adbeb4f0936fa23 (127.21.54.131:38839): Not found: Tablet not found: 007363f9b3384029b77d284b4fef93b4
W20260811 17:16:58.168390 6789 leader_election.cc:341] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8692e983abb54c31bced385540a68fee (127.21.54.130:45421): Not found: Tablet not found: 007363f9b3384029b77d284b4fef93b4
I20260811 17:16:58.168466 6789 leader_election.cc:302] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da6b2e8a85c64c859170faef63149240; no voters: 8692e983abb54c31bced385540a68fee, d0e1c0675ae649f88adbeb4f0936fa23
I20260811 17:16:58.168668 7252 raft_consensus.cc:2726] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:16:58.635234 7253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:58.634741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:52839)
0811 17:16:58.635062 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:58.635130 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":93,"threads_started":1}
I20260811 17:16:58.769593 7254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:58.768945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:33291)
0811 17:16:58.769323 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:58.769383 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":116,"threads_started":1}
I20260811 17:16:58.970742 7255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:58.970248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:45305)
0811 17:16:58.970575 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:58.970635 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":175,"threads_started":1}
I20260811 17:16:59.636834 7256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:59.636280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:60275)
0811 17:16:59.636683 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:59.636736 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":106,"threads_started":1}
I20260811 17:16:59.770867 7257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:59.770384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:54105)
0811 17:16:59.770705 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:59.770764 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":89,"threads_started":1}
I20260811 17:16:59.972213 7258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:16:59.971654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:60785)
0811 17:16:59.972031 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0811 17:16:59.972102 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":100,"threads_started":1}
I20260811 17:17:00.638154 7259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:00.637681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:41285)
0811 17:17:00.637978 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:00.638046 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":81,"threads_started":1}
I20260811 17:17:00.772207 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:00.771770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:38899)
0811 17:17:00.772044 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:00.772104 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":96,"threads_started":1}
I20260811 17:17:00.973642 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:00.973164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:51647)
0811 17:17:00.973466 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:00.973537 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":101,"threads_started":1}
I20260811 17:17:01.514070 7262 raft_consensus.cc:482] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:17:01.514214 7262 raft_consensus.cc:504] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } }
I20260811 17:17:01.514801 7262 leader_election.cc:288] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d0e1c0675ae649f88adbeb4f0936fa23 (127.21.54.131:38839), 8692e983abb54c31bced385540a68fee (127.21.54.130:45421)
W20260811 17:17:01.515688 6789 leader_election.cc:341] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8692e983abb54c31bced385540a68fee (127.21.54.130:45421): Not found: Tablet not found: 007363f9b3384029b77d284b4fef93b4
W20260811 17:17:01.515848 6792 leader_election.cc:341] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d0e1c0675ae649f88adbeb4f0936fa23 (127.21.54.131:38839): Not found: Tablet not found: 007363f9b3384029b77d284b4fef93b4
I20260811 17:17:01.515924 6792 leader_election.cc:302] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da6b2e8a85c64c859170faef63149240; no voters: 8692e983abb54c31bced385540a68fee, d0e1c0675ae649f88adbeb4f0936fa23
I20260811 17:17:01.516106 7262 raft_consensus.cc:2726] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:17:01.639628 7263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:01.639116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:44809)
0811 17:17:01.639400 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:01.639514 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":85,"threads_started":1}
I20260811 17:17:01.773540 7264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:01.773015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:56473)
0811 17:17:01.773324 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:01.773383 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":78,"threads_started":1}
I20260811 17:17:01.975276 7265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:01.974764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:55601)
0811 17:17:01.975096 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:01.975169 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":90,"threads_started":1}
I20260811 17:17:02.641356 7266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:02.640860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:57413)
0811 17:17:02.641175 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:02.641250 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":107,"threads_started":1}
I20260811 17:17:02.774955 7267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:02.774475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:35283)
0811 17:17:02.774783 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:02.774848 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":107,"threads_started":1}
I20260811 17:17:02.976930 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:02.976354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:38515)
0811 17:17:02.976757 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:02.976823 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":108,"threads_started":1}
I20260811 17:17:03.642647 7270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:03.642205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:59585)
0811 17:17:03.642470 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:03.642534 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":82,"threads_started":1}
I20260811 17:17:03.776268 7271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:03.775827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:40087)
0811 17:17:03.776103 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:03.776165 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":76,"threads_started":1}
I20260811 17:17:03.978564 7272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:03.978007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:45935)
0811 17:17:03.978349 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:03.978409 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":111,"threads_started":1}
I20260811 17:17:04.644086 7273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:04.643598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:34779)
0811 17:17:04.643899 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:04.643974 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":86,"threads_started":1}
I20260811 17:17:04.777585 7275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:04.777148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:55083)
0811 17:17:04.777420 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:04.777478 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":88,"threads_started":1}
I20260811 17:17:04.980046 7276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:04.979483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:60029)
0811 17:17:04.979855 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:04.979933 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":85,"threads_started":1}
I20260811 17:17:05.645437 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:05.644963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:58437)
0811 17:17:05.645271 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:05.645335 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":92,"threads_started":1}
I20260811 17:17:05.779073 7278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:05.778499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:60105)
0811 17:17:05.778874 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:05.778962 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":154,"threads_started":1}
I20260811 17:17:05.947561 7279 raft_consensus.cc:482] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:17:05.947717 7279 raft_consensus.cc:504] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e1c0675ae649f88adbeb4f0936fa23" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 38839 } } peers { permanent_uuid: "8692e983abb54c31bced385540a68fee" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45421 } } peers { permanent_uuid: "da6b2e8a85c64c859170faef63149240" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37467 } }
I20260811 17:17:05.948159 7279 leader_election.cc:288] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d0e1c0675ae649f88adbeb4f0936fa23 (127.21.54.131:38839), 8692e983abb54c31bced385540a68fee (127.21.54.130:45421)
W20260811 17:17:05.949142 6789 leader_election.cc:341] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8692e983abb54c31bced385540a68fee (127.21.54.130:45421): Not found: Tablet not found: 007363f9b3384029b77d284b4fef93b4
W20260811 17:17:05.949257 6792 leader_election.cc:341] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d0e1c0675ae649f88adbeb4f0936fa23 (127.21.54.131:38839): Not found: Tablet not found: 007363f9b3384029b77d284b4fef93b4
I20260811 17:17:05.949376 6792 leader_election.cc:302] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da6b2e8a85c64c859170faef63149240; no voters: 8692e983abb54c31bced385540a68fee, d0e1c0675ae649f88adbeb4f0936fa23
I20260811 17:17:05.949640 7279 raft_consensus.cc:2726] T 007363f9b3384029b77d284b4fef93b4 P da6b2e8a85c64c859170faef63149240 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:17:05.981498 7280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:05.981031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:54557)
0811 17:17:05.981322 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:05.981391 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":88,"threads_started":1}
I20260811 17:17:06.646903 7281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:06.646399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:42811)
0811 17:17:06.646731 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:06.646795 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":87,"threads_started":1}
I20260811 17:17:06.780395 7282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:06.779960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:36795)
0811 17:17:06.780237 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:06.780292 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":80,"threads_started":1}
I20260811 17:17:06.982896 7283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:06.982425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:44215)
0811 17:17:06.982712 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:06.982778 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":85,"threads_started":1}
I20260811 17:17:07.648595 7284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:07.648038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.129:49291)
0811 17:17:07.648344 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:07.648407 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":85,"threads_started":1}
I20260811 17:17:07.782380 7285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:07.781826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.130:33403)
0811 17:17:07.782185 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:07.782266 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":111,"threads_started":1}
I20260811 17:17:07.965975 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 6734
I20260811 17:17:07.978029 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 6922
I20260811 17:17:07.984855 7286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:07.984359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:45313 (local address 127.21.54.131:49441)
0811 17:17:07.984692 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:07.984751 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:45313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":101,"threads_started":1}
I20260811 17:17:07.990156 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 7078
2026-08-11T17:17:07Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19596 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:17:08 dist-test-slave-fd7l krb5kdc[7289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:17:08 dist-test-slave-fd7l krb5kdc[7289](info): set up 2 sockets
Aug 11 17:17:08 dist-test-slave-fd7l krb5kdc[7289](info): commencing operation
krb5kdc: starting...
W20260811 17:17:09.184333 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:17:09 dist-test-slave-fd7l krb5kdc[7289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:17:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:17:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:17:09.331586 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:43111
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:43111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:17:09.463649 7304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:17:09.464187 7304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:17:09.464319 7304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:17:09.471750 7304 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 17:17:09.471838 7304 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 17:17:09.471859 7304 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 17:17:09.471877 7304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:17:09.471904 7304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:17:09.471969 7304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:17:09.472060 7304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:17:09.472108 7304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:17:09.478447 7304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:43111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:43111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:17:09.479737 7304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:17:09.480932 7304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:17:09.491040 7310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:09.491079 7309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:09.491088 7315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:17:09.491078 7304 server_base.cc:1034] running on GCE node
I20260811 17:17:09.492023 7304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:17:09.493430 7304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:17:09.494666 7304 hybrid_clock.cc:648] HybridClock initialized: now 1786468629494626 us; error 53 us; skew 500 ppm
Aug 11 17:17:09 dist-test-slave-fd7l krb5kdc[7289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468629, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:17:09.498250 7304 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:17:09.499298 7304 webserver.cc:460] Webserver started at http://127.21.54.190:39081/ using document root <none> and password file <none>
I20260811 17:17:09.499917 7304 fs_manager.cc:359] Metadata directory not provided
I20260811 17:17:09.499984 7304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:17:09.500172 7304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:17:09.502539 7304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "4b9e0859907b48038085e86883d06448"
format_stamp: "Formatted at 2026-08-11 17:17:09 on dist-test-slave-fd7l"
server_key: "3a8c39bdde802ffdda4afa28feb40c1d"
server_key_iv: "fddf60176ba3d5aba2333e91e121c668"
server_key_version: "encryptionkey@0"
I20260811 17:17:09.503109 7304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "4b9e0859907b48038085e86883d06448"
format_stamp: "Formatted at 2026-08-11 17:17:09 on dist-test-slave-fd7l"
server_key: "3a8c39bdde802ffdda4afa28feb40c1d"
server_key_iv: "fddf60176ba3d5aba2333e91e121c668"
server_key_version: "encryptionkey@0"
I20260811 17:17:09.507481 7304 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:17:09.510485 7322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:09.511933 7304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:17:09.512115 7304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "4b9e0859907b48038085e86883d06448"
format_stamp: "Formatted at 2026-08-11 17:17:09 on dist-test-slave-fd7l"
server_key: "3a8c39bdde802ffdda4afa28feb40c1d"
server_key_iv: "fddf60176ba3d5aba2333e91e121c668"
server_key_version: "encryptionkey@0"
I20260811 17:17:09.512254 7304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:17:09.533529 7304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:17:09.537261 7304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:17:09.537515 7304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:17:09.545400 7304 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:43111
I20260811 17:17:09.545773 7377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:43111 every 8 connection(s)
I20260811 17:17:09.546465 7304 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:17:09.549466 7378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:17:09.550743 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 7304
I20260811 17:17:09.550971 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:17:09.551326 21722 external_mini_cluster.cc:1428] Setting key 10a61397f4aa05d7f060d002d49e2637
I20260811 17:17:09.556070 7378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448: Bootstrap starting.
I20260811 17:17:09.558727 7378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448: Neither blocks nor log segments found. Creating new log.
I20260811 17:17:09.559660 7378 log.cc:824] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448: Log is configured to *not* fsync() on all Append() calls
Aug 11 17:17:09 dist-test-slave-fd7l krb5kdc[7289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:17:09.562052 7378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448: No bootstrap required, opened a new log
I20260811 17:17:09.565835 7378 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9e0859907b48038085e86883d06448" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43111 } }
I20260811 17:17:09.566104 7378 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:17:09.566193 7378 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b9e0859907b48038085e86883d06448, State: Initialized, Role: FOLLOWER
I20260811 17:17:09.566721 7378 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9e0859907b48038085e86883d06448" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43111 } }
I20260811 17:17:09.566879 7378 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:17:09.566970 7378 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:17:09.567091 7378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:17:09.568140 7378 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9e0859907b48038085e86883d06448" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43111 } }
I20260811 17:17:09.568146 7381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:09.552954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59826 (local address 127.21.54.190:43111)
0811 17:17:09.553440 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:09.553458 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:09.553497 (+ 39us) server_negotiation.cc:407] Connection header received
0811 17:17:09.554228 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:09.554247 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:09.554609 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:09.554916 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:17:09.555903 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:09.557222 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:09.558377 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:09.559001 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:09.559158 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:09.562233 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:17:09.562239 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:17:09.562257 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:17:09.562299 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:17:09.564307 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:09.564915 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:09.564924 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:09.564929 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:09.565106 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:09.565471 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:09.565477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:09.565479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:09.565944 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:17:09.566202 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:09.566922 (+ 720us) server_negotiation.cc:299] Negotiation successful
0811 17:17:09.567305 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":158,"threads_started":1}
I20260811 17:17:09.568622 7378 leader_election.cc:302] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b9e0859907b48038085e86883d06448; no voters:
I20260811 17:17:09.568981 7378 leader_election.cc:288] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:17:09.569132 7383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:17:09.569402 7383 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 1 LEADER]: Becoming Leader. State: Replica: 4b9e0859907b48038085e86883d06448, State: Running, Role: LEADER
I20260811 17:17:09.569794 7383 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9e0859907b48038085e86883d06448" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43111 } }
I20260811 17:17:09.571079 7383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b9e0859907b48038085e86883d06448. Latest consensus state: current_term: 1 leader_uuid: "4b9e0859907b48038085e86883d06448" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9e0859907b48038085e86883d06448" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43111 } } }
I20260811 17:17:09.571323 7383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [sys.catalog]: This master's current role is: LEADER
I20260811 17:17:09.571374 7378 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:17:09.571604 7386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b9e0859907b48038085e86883d06448" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9e0859907b48038085e86883d06448" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43111 } } }
I20260811 17:17:09.571708 7386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [sys.catalog]: This master's current role is: LEADER
I20260811 17:17:09.573154 7388 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:17:09.576340 7388 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:17:09.583079 7388 catalog_manager.cc:1347] Generated new cluster ID: bae271c9dfcf402ab74ac3570fde225b
I20260811 17:17:09.583207 7388 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:17:09.607909 7388 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:17:09.609179 7388 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:17:09.617635 7388 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448: Generated new TSK 0
I20260811 17:17:09.618468 7388 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
I20260811 17:17:09.664171 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:43111
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:17:09.784408 7408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:17:09.784732 7408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:17:09.784777 7408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:17:09.788753 7408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:17:09.788811 7408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:17:09.788870 7408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:17:09.794286 7408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:43111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:17:09.795390 7408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:17:09.796401 7408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:17:09.806854 7414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:09.806864 7413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:09.806869 7418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:17:09.807272 7408 server_base.cc:1034] running on GCE node
I20260811 17:17:09.807777 7408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:17:09.808467 7408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:17:09.809723 7408 hybrid_clock.cc:648] HybridClock initialized: now 1786468629809687 us; error 47 us; skew 500 ppm
Aug 11 17:17:09 dist-test-slave-fd7l krb5kdc[7289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468629, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:17:09.813014 7408 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:17:09.814052 7408 webserver.cc:460] Webserver started at http://127.21.54.129:35165/ using document root <none> and password file <none>
I20260811 17:17:09.814790 7408 fs_manager.cc:359] Metadata directory not provided
I20260811 17:17:09.814905 7408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:17:09.815179 7408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:17:09.817902 7408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "51791ee80155440d901587802cb71758"
format_stamp: "Formatted at 2026-08-11 17:17:09 on dist-test-slave-fd7l"
server_key: "e1244db4fb83633b37b469f8289ca898"
server_key_iv: "accf92158196876faa4ae1f819fd5f1a"
server_key_version: "encryptionkey@0"
I20260811 17:17:09.818581 7408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "51791ee80155440d901587802cb71758"
format_stamp: "Formatted at 2026-08-11 17:17:09 on dist-test-slave-fd7l"
server_key: "e1244db4fb83633b37b469f8289ca898"
server_key_iv: "accf92158196876faa4ae1f819fd5f1a"
server_key_version: "encryptionkey@0"
I20260811 17:17:09.823160 7408 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:17:09.826289 7425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:09.827669 7408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:17:09.827826 7408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "51791ee80155440d901587802cb71758"
format_stamp: "Formatted at 2026-08-11 17:17:09 on dist-test-slave-fd7l"
server_key: "e1244db4fb83633b37b469f8289ca898"
server_key_iv: "accf92158196876faa4ae1f819fd5f1a"
server_key_version: "encryptionkey@0"
I20260811 17:17:09.827960 7408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:17:09.843930 7408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:17:09.847163 7408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:17:09.847389 7408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:17:09.848042 7408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:17:09.848989 7408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:17:09.849068 7408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:09.849144 7408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:17:09.849196 7408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:09.859376 7408 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:45361
I20260811 17:17:09.859424 7560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:45361 every 8 connection(s)
I20260811 17:17:09.860390 7408 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:17:09.862818 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 7408
I20260811 17:17:09.863003 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:17:09.863361 21722 external_mini_cluster.cc:1428] Setting key cb0e679ed1a949111d9e43d202b682b2
Aug 11 17:17:09 dist-test-slave-fd7l krb5kdc[7289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468629, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.54.130@KRBTEST.COM; defaulting to no policy
I20260811 17:17:09.877506 7381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:09.862538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:59469 (local address 127.21.54.190:43111)
0811 17:17:09.862793 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:09.862801 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:09.863772 (+ 971us) server_negotiation.cc:407] Connection header received
0811 17:17:09.865645 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:09.865652 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:09.865715 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:09.865804 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:17:09.867072 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:09.867718 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:09.869598 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:09.869835 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:09.869921 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:09.873478 (+ 3557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:17:09.873481 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:17:09.873483 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:17:09.873511 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:17:09.875112 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:09.875698 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:09.875701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:09.875705 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:09.875791 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:09.876238 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:09.876241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:09.876242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:09.876448 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:17:09.876616 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:09.877183 (+ 567us) server_negotiation.cc:299] Negotiation successful
0811 17:17:09.877330 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.130@KRBTEST.COM" created.
I20260811 17:17:09.878477 7563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:09.862946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:59469)
0811 17:17:09.863573 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:09.863636 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 17:17:09.865376 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:17:09.866002 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:17:09.866009 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:17:09.866387 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:17:09.866906 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:09.866920 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:09.867863 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:09.867868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:09.869434 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:09.869450 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:09.870021 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:09.870028 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:09.870322 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:09.871062 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:17:09.871086 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:17:09.873264 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:17:09.875243 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:17:09.875250 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:17:09.875269 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:17:09.875577 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:17:09.875904 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:17:09.875907 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:17:09.875909 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:17:09.876117 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:17:09.876616 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:17:09.876625 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:17:09.876921 (+ 296us) client_negotiation.cc:769] Sending connection context
0811 17:17:09.877129 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 17:17:09.877503 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":140,"threads_started":1}
I20260811 17:17:09.880069 7561 heartbeater.cc:347] Connected to a master server at 127.21.54.190:43111
I20260811 17:17:09.880352 7561 heartbeater.cc:464] Registering TS with master...
I20260811 17:17:09.880996 7561 heartbeater.cc:511] Master 127.21.54.190:43111 requested a full tablet report, sending...
I20260811 17:17:09.882651 7340 ts_manager.cc:194] Registered new tserver with Master: 51791ee80155440d901587802cb71758 (127.21.54.129:45361)
I20260811 17:17:09.883841 7340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:59469
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
I20260811 17:17:09.899888 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:43111
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:17:10.020442 7567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:17:10.020793 7567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:17:10.020838 7567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:17:10.024907 7567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:17:10.024971 7567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:17:10.025030 7567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:17:10.030462 7567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:43111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:17:10.031764 7567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:17:10.032991 7567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:17:10.043360 7576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:10.043422 7574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:10.043360 7573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:17:10.043401 7567 server_base.cc:1034] running on GCE node
I20260811 17:17:10.044138 7567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:17:10.044884 7567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:17:10.046072 7567 hybrid_clock.cc:648] HybridClock initialized: now 1786468630046016 us; error 67 us; skew 500 ppm
Aug 11 17:17:10 dist-test-slave-fd7l krb5kdc[7289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468630, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:17:10.049216 7567 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:17:10.050179 7567 webserver.cc:460] Webserver started at http://127.21.54.130:46137/ using document root <none> and password file <none>
I20260811 17:17:10.050783 7567 fs_manager.cc:359] Metadata directory not provided
I20260811 17:17:10.050848 7567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:17:10.051028 7567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:17:10.053611 7567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "29b38e0787754d12a0ee28ed380cfd0d"
format_stamp: "Formatted at 2026-08-11 17:17:10 on dist-test-slave-fd7l"
server_key: "300f8ee1db0729a9b5954a69410675a4"
server_key_iv: "4f07085aee45e845441f1dfb6a66d82a"
server_key_version: "encryptionkey@0"
I20260811 17:17:10.054189 7567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "29b38e0787754d12a0ee28ed380cfd0d"
format_stamp: "Formatted at 2026-08-11 17:17:10 on dist-test-slave-fd7l"
server_key: "300f8ee1db0729a9b5954a69410675a4"
server_key_iv: "4f07085aee45e845441f1dfb6a66d82a"
server_key_version: "encryptionkey@0"
I20260811 17:17:10.058339 7567 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 17:17:10.060938 7587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:10.062292 7567 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:17:10.062450 7567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "29b38e0787754d12a0ee28ed380cfd0d"
format_stamp: "Formatted at 2026-08-11 17:17:10 on dist-test-slave-fd7l"
server_key: "300f8ee1db0729a9b5954a69410675a4"
server_key_iv: "4f07085aee45e845441f1dfb6a66d82a"
server_key_version: "encryptionkey@0"
I20260811 17:17:10.062559 7567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:17:10.080269 7567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:17:10.083709 7567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:17:10.083920 7567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:17:10.084611 7567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:17:10.085518 7567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:17:10.085592 7567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:10.085678 7567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:17:10.085727 7567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:10.095088 7567 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:42201
I20260811 17:17:10.095136 7719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:42201 every 8 connection(s)
I20260811 17:17:10.096094 7567 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:17:10.098364 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 7567
I20260811 17:17:10.098505 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:17:10.098804 21722 external_mini_cluster.cc:1428] Setting key 1a25a4cbf12d03839fbf60436b2c5f8e
Aug 11 17:17:10 dist-test-slave-fd7l krb5kdc[7289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468630, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.54.131@KRBTEST.COM; defaulting to no policy
I20260811 17:17:10.112196 7381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:10.098227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:44159 (local address 127.21.54.190:43111)
0811 17:17:10.098395 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:10.098401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:10.099596 (+ 1195us) server_negotiation.cc:407] Connection header received
0811 17:17:10.101124 (+ 1528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:10.101129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:10.101192 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:10.101290 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:17:10.102639 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.103276 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.104470 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.104773 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.104892 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:10.107507 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:17:10.107512 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:17:10.107517 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:17:10.107554 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:17:10.109536 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:10.110218 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:10.110225 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:10.110229 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:10.110335 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:10.110837 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:10.110840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:10.110841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:10.111074 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:17:10.111219 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:10.111930 (+ 711us) server_negotiation.cc:299] Negotiation successful
0811 17:17:10.112067 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 17:17:10.113204 7722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:10.098669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:44159)
0811 17:17:10.099235 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:10.099298 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 17:17:10.100900 (+ 1602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:17:10.101452 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:17:10.101458 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:17:10.101933 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:17:10.102454 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:10.102468 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.103413 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.103416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:10.104333 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:10.104342 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.104936 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.104941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:10.105134 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:10.105777 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:17:10.105795 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:17:10.107310 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:17:10.109683 (+ 2373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:17:10.109689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:17:10.109703 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:17:10.110105 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:17:10.110473 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:17:10.110476 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:17:10.110478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:17:10.110699 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:17:10.111203 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:17:10.111209 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:17:10.111626 (+ 417us) client_negotiation.cc:769] Sending connection context
0811 17:17:10.111812 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 17:17:10.112108 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":150,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.131@KRBTEST.COM" created.
I20260811 17:17:10.114543 7720 heartbeater.cc:347] Connected to a master server at 127.21.54.190:43111
I20260811 17:17:10.114832 7720 heartbeater.cc:464] Registering TS with master...
I20260811 17:17:10.115514 7720 heartbeater.cc:511] Master 127.21.54.190:43111 requested a full tablet report, sending...
I20260811 17:17:10.116945 7340 ts_manager.cc:194] Registered new tserver with Master: 29b38e0787754d12a0ee28ed380cfd0d (127.21.54.130:42201)
I20260811 17:17:10.117641 7340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:44159
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
I20260811 17:17:10.132483 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:43111
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:17:10.255939 7725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:17:10.256240 7725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:17:10.256286 7725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:17:10.260500 7725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:17:10.260599 7725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:17:10.260663 7725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:17:10.266450 7725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:43111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:17:10.267805 7725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:17:10.268953 7725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:17:10.279946 7733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:10.279968 7731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:17:10.280409 7725 server_base.cc:1034] running on GCE node
W20260811 17:17:10.279970 7730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:17:10.281032 7725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:17:10.281767 7725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:17:10.282979 7725 hybrid_clock.cc:648] HybridClock initialized: now 1786468630282962 us; error 32 us; skew 500 ppm
Aug 11 17:17:10 dist-test-slave-fd7l krb5kdc[7289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468630, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:17:10.286152 7725 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:17:10.287161 7725 webserver.cc:460] Webserver started at http://127.21.54.131:46243/ using document root <none> and password file <none>
I20260811 17:17:10.287729 7725 fs_manager.cc:359] Metadata directory not provided
I20260811 17:17:10.287791 7725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:17:10.287971 7725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:17:10.290208 7725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "7a3eef0fd45f40b783d298ce46aee818"
format_stamp: "Formatted at 2026-08-11 17:17:10 on dist-test-slave-fd7l"
server_key: "7005e6ad7aba23ea0f06438f031d5f2e"
server_key_iv: "1df7625b7e605e91be196d54ab2eef42"
server_key_version: "encryptionkey@0"
I20260811 17:17:10.290776 7725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "7a3eef0fd45f40b783d298ce46aee818"
format_stamp: "Formatted at 2026-08-11 17:17:10 on dist-test-slave-fd7l"
server_key: "7005e6ad7aba23ea0f06438f031d5f2e"
server_key_iv: "1df7625b7e605e91be196d54ab2eef42"
server_key_version: "encryptionkey@0"
I20260811 17:17:10.295001 7725 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:17:10.298157 7741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:10.299837 7725 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:17:10.299964 7725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "7a3eef0fd45f40b783d298ce46aee818"
format_stamp: "Formatted at 2026-08-11 17:17:10 on dist-test-slave-fd7l"
server_key: "7005e6ad7aba23ea0f06438f031d5f2e"
server_key_iv: "1df7625b7e605e91be196d54ab2eef42"
server_key_version: "encryptionkey@0"
I20260811 17:17:10.300135 7725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:17:10.311657 7725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:17:10.315189 7725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:17:10.315423 7725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:17:10.316165 7725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:17:10.317266 7725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:17:10.317332 7725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:10.317396 7725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:17:10.317430 7725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:10.326826 7725 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:40579
I20260811 17:17:10.326928 7870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:40579 every 8 connection(s)
I20260811 17:17:10.327847 7725 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:17:10.331630 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 7725
I20260811 17:17:10.331799 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:17:10.332191 21722 external_mini_cluster.cc:1428] Setting key 5a2fcc87509009c0252c69a529377504
Aug 11 17:17:10 dist-test-slave-fd7l krb5kdc[7289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468630, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:17:10.343567 7381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:10.330011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:40723 (local address 127.21.54.190:43111)
0811 17:17:10.330185 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:10.330189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:10.331171 (+ 982us) server_negotiation.cc:407] Connection header received
0811 17:17:10.333039 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:10.333043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:10.333108 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:10.333210 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:17:10.334883 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.335446 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.336742 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.336992 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.337095 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:10.339853 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:17:10.339856 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:17:10.339859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:17:10.339886 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:17:10.341378 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:10.341970 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:10.341973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:10.341974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:10.342066 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:10.342413 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:10.342416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:10.342417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:10.342668 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:17:10.342757 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:10.343276 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 17:17:10.343390 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 17:17:10.344405 7873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:10.330412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:40723)
0811 17:17:10.330992 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:10.331049 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 17:17:10.332728 (+ 1679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:17:10.333450 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:17:10.333459 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:17:10.334058 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:17:10.334685 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:10.334707 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.335611 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.335613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:10.336595 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:10.336606 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.337172 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.337176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:10.337346 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:10.338080 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:17:10.338098 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:17:10.339667 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:17:10.341506 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:17:10.341512 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:17:10.341566 (+ 54us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:17:10.341843 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:17:10.342169 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:17:10.342171 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:17:10.342172 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:17:10.342324 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:17:10.342766 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:17:10.342771 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:17:10.343046 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 17:17:10.343215 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 17:17:10.343494 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":134,"threads_started":1}
I20260811 17:17:10.345963 7871 heartbeater.cc:347] Connected to a master server at 127.21.54.190:43111
I20260811 17:17:10.346251 7871 heartbeater.cc:464] Registering TS with master...
I20260811 17:17:10.346920 7871 heartbeater.cc:511] Master 127.21.54.190:43111 requested a full tablet report, sending...
I20260811 17:17:10.348083 7340 ts_manager.cc:194] Registered new tserver with Master: 7a3eef0fd45f40b783d298ce46aee818 (127.21.54.131:40579)
I20260811 17:17:10.348763 7340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:40723
I20260811 17:17:10.359684 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:17:10.370256 7381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:10.361849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59832 (local address 127.21.54.190:43111)
0811 17:17:10.362018 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:10.362024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:10.362228 (+ 204us) server_negotiation.cc:407] Connection header received
0811 17:17:10.362400 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:10.362404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:10.362459 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:10.362545 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:17:10.363333 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.363872 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.364811 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.365046 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.365175 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:10.366746 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:17:10.366750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:17:10.366752 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:17:10.366801 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:17:10.368401 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:10.368920 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:10.368925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:10.368927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:10.369019 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:10.369314 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:10.369317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:10.369319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:10.369562 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:17:10.369711 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:10.369958 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 17:17:10.370095 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 17:17:10.373701 7340 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:17:10.377084 7340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:17:10.393873 7885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:10.387095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:42201 (local address 127.0.0.1:46664)
0811 17:17:10.387774 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:10.387793 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:17:10.387930 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:17:10.389218 (+ 1288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:17:10.389223 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:17:10.389245 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:17:10.389346 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:10.389353 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.390625 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.390632 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:10.391963 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:10.391976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.393205 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.393210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:10.393335 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:10.393407 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 17:17:10.393533 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 17:17:10.393694 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":57,"thread_start_us":122,"threads_started":1}
I20260811 17:17:10.394019 7888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:10.388227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46664 (local address 127.21.54.130:42201)
0811 17:17:10.388735 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:10.388749 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:10.388774 (+ 25us) server_negotiation.cc:407] Connection header received
0811 17:17:10.388862 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:10.388868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:10.389074 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:10.389250 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:17:10.389466 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.390428 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.392113 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.393044 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.393183 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:10.393375 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:10.393723 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 17:17:10.393837 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":138,"threads_started":1}
I20260811 17:17:10.394291 7887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:10.388248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48628 (local address 127.21.54.129:45361)
0811 17:17:10.388735 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:10.388745 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:10.388774 (+ 29us) server_negotiation.cc:407] Connection header received
0811 17:17:10.388862 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:10.388868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:10.389085 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:10.389250 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:17:10.389751 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.390956 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.392632 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.393307 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.393423 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:10.393616 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:10.393981 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 17:17:10.394114 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":132,"threads_started":1}
I20260811 17:17:10.394613 7886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:10.387739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:45361 (local address 127.0.0.1:48628)
0811 17:17:10.388458 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:10.388479 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 17:17:10.388613 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:17:10.389448 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:17:10.389452 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:17:10.389475 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:17:10.389579 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:10.389587 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.391168 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.391171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:10.392464 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:10.392474 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.393482 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.393487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:10.393619 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:10.393652 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 17:17:10.393742 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 17:17:10.393829 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":112,"threads_started":1}
I20260811 17:17:10.395013 7883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:10.386797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:40579 (local address 127.0.0.1:40248)
0811 17:17:10.387724 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:10.387783 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 17:17:10.387925 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:17:10.388880 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:17:10.388885 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:17:10.388906 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:17:10.389029 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:10.389034 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.390656 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.390660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:10.392344 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:10.392356 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.394597 (+ 2241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.394601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:10.394704 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:10.394731 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 17:17:10.394802 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 17:17:10.394876 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":88,"thread_start_us":106,"threads_started":1}
I20260811 17:17:10.395269 7884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:10.386996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40248 (local address 127.21.54.131:40579)
0811 17:17:10.387871 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:10.387888 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:10.387922 (+ 34us) server_negotiation.cc:407] Connection header received
0811 17:17:10.388172 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:10.388180 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:10.388429 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:10.388698 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:17:10.389171 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.390485 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.392542 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.393506 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.393638 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:10.394911 (+ 1273us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:10.394979 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 17:17:10.395093 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":147,"threads_started":1}
I20260811 17:17:10.397686 7477 tablet_service.cc:1467] Processing CreateTablet for tablet 2ed6815d740948bba31c5976f0b1f26b (DEFAULT_TABLE table=test-table [id=a0b13a2b76554ff79dfbbf4f8af08fe1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:17:10.397852 7639 tablet_service.cc:1467] Processing CreateTablet for tablet 2ed6815d740948bba31c5976f0b1f26b (DEFAULT_TABLE table=test-table [id=a0b13a2b76554ff79dfbbf4f8af08fe1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:17:10.397979 7794 tablet_service.cc:1467] Processing CreateTablet for tablet 2ed6815d740948bba31c5976f0b1f26b (DEFAULT_TABLE table=test-table [id=a0b13a2b76554ff79dfbbf4f8af08fe1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:17:10.399088 7794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ed6815d740948bba31c5976f0b1f26b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:17:10.399084 7639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ed6815d740948bba31c5976f0b1f26b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:17:10.399086 7477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ed6815d740948bba31c5976f0b1f26b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:17:10.406838 7890 tablet_bootstrap.cc:492] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758: Bootstrap starting.
I20260811 17:17:10.408530 7891 tablet_bootstrap.cc:492] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d: Bootstrap starting.
I20260811 17:17:10.408530 7892 tablet_bootstrap.cc:492] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818: Bootstrap starting.
I20260811 17:17:10.409837 7890 tablet_bootstrap.cc:654] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758: Neither blocks nor log segments found. Creating new log.
I20260811 17:17:10.410876 7890 log.cc:824] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758: Log is configured to *not* fsync() on all Append() calls
I20260811 17:17:10.411060 7892 tablet_bootstrap.cc:654] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818: Neither blocks nor log segments found. Creating new log.
I20260811 17:17:10.411659 7891 tablet_bootstrap.cc:654] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d: Neither blocks nor log segments found. Creating new log.
I20260811 17:17:10.412616 7891 log.cc:824] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d: Log is configured to *not* fsync() on all Append() calls
I20260811 17:17:10.412616 7892 log.cc:824] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818: Log is configured to *not* fsync() on all Append() calls
I20260811 17:17:10.413450 7890 tablet_bootstrap.cc:492] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758: No bootstrap required, opened a new log
I20260811 17:17:10.413715 7890 ts_tablet_manager.cc:1397] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:17:10.414923 7892 tablet_bootstrap.cc:492] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818: No bootstrap required, opened a new log
I20260811 17:17:10.414918 7891 tablet_bootstrap.cc:492] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d: No bootstrap required, opened a new log
I20260811 17:17:10.415138 7891 ts_tablet_manager.cc:1397] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:17:10.415138 7892 ts_tablet_manager.cc:1397] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:17:10.418964 7890 raft_consensus.cc:348] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:10.419303 7890 raft_consensus.cc:374] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:17:10.419374 7890 raft_consensus.cc:729] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51791ee80155440d901587802cb71758, State: Initialized, Role: FOLLOWER
I20260811 17:17:10.419564 7891 raft_consensus.cc:348] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:10.419564 7892 raft_consensus.cc:348] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:10.419884 7891 raft_consensus.cc:374] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:17:10.419884 7892 raft_consensus.cc:374] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:17:10.419952 7892 raft_consensus.cc:729] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a3eef0fd45f40b783d298ce46aee818, State: Initialized, Role: FOLLOWER
I20260811 17:17:10.419960 7891 raft_consensus.cc:729] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29b38e0787754d12a0ee28ed380cfd0d, State: Initialized, Role: FOLLOWER
I20260811 17:17:10.420015 7890 consensus_queue.cc:260] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:10.420473 7892 consensus_queue.cc:260] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:10.420471 7891 consensus_queue.cc:260] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:10.421099 7561 heartbeater.cc:503] Master 127.21.54.190:43111 was elected leader, sending a full tablet report...
I20260811 17:17:10.421370 7890 ts_tablet_manager.cc:1428] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260811 17:17:10.421591 7892 ts_tablet_manager.cc:1428] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 17:17:10.421586 7891 ts_tablet_manager.cc:1428] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 17:17:10.421954 7720 heartbeater.cc:503] Master 127.21.54.190:43111 was elected leader, sending a full tablet report...
I20260811 17:17:10.422904 7871 heartbeater.cc:503] Master 127.21.54.190:43111 was elected leader, sending a full tablet report...
W20260811 17:17:10.578970 7872 tablet.cc:2369] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:17:10.597609 7721 tablet.cc:2369] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:17:10.611949 7562 tablet.cc:2369] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:17:10.733747 7898 raft_consensus.cc:482] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:17:10.734030 7898 raft_consensus.cc:504] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:10.735308 7898 leader_election.cc:288] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a3eef0fd45f40b783d298ce46aee818 (127.21.54.131:40579), 51791ee80155440d901587802cb71758 (127.21.54.129:45361)
I20260811 17:17:10.741273 7884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:10.735796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:49847 (local address 127.21.54.131:40579)
0811 17:17:10.735920 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:10.735925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:10.736292 (+ 367us) server_negotiation.cc:407] Connection header received
0811 17:17:10.736352 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:10.736355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:10.736403 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:10.736721 (+ 318us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:17:10.736934 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.737829 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.739708 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.740420 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.740630 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:10.740692 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:10.741007 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 17:17:10.741130 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:17:10.741273 7887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:10.735757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:50663 (local address 127.21.54.129:45361)
0811 17:17:10.735902 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:10.735907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:10.736206 (+ 299us) server_negotiation.cc:407] Connection header received
0811 17:17:10.736343 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:10.736346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:10.736403 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:10.736536 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:17:10.736957 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.737823 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.739708 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.740450 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.740631 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:10.740692 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:10.741021 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 17:17:10.741141 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 17:17:10.741285 7899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:10.735632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:45361 (local address 127.21.54.130:50663)
0811 17:17:10.736056 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:10.736083 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 17:17:10.736231 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:17:10.736565 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:17:10.736578 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:17:10.736604 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:17:10.736750 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:10.736760 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.737969 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.737972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:10.739506 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:10.739519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.740657 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.740659 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:10.740825 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:10.740859 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 17:17:10.740945 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 17:17:10.741072 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":224,"thread_start_us":99,"threads_started":1}
I20260811 17:17:10.741283 7900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:10.735639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:40579 (local address 127.21.54.130:49847)
0811 17:17:10.736171 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:10.736184 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:17:10.736273 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:17:10.736672 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:17:10.736675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:17:10.736687 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:17:10.736788 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:10.736795 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.737969 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.737972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:10.739506 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:10.739519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.740631 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.740636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:10.740787 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:10.740863 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 17:17:10.740954 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 17:17:10.741066 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":232,"thread_start_us":66,"threads_started":1}
I20260811 17:17:10.742108 7818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ed6815d740948bba31c5976f0b1f26b" candidate_uuid: "29b38e0787754d12a0ee28ed380cfd0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a3eef0fd45f40b783d298ce46aee818" is_pre_election: true
I20260811 17:17:10.742376 7504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ed6815d740948bba31c5976f0b1f26b" candidate_uuid: "29b38e0787754d12a0ee28ed380cfd0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51791ee80155440d901587802cb71758" is_pre_election: true
I20260811 17:17:10.742493 7818 raft_consensus.cc:2454] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29b38e0787754d12a0ee28ed380cfd0d in term 0.
I20260811 17:17:10.742718 7504 raft_consensus.cc:2454] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29b38e0787754d12a0ee28ed380cfd0d in term 0.
I20260811 17:17:10.743165 7589 leader_election.cc:302] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29b38e0787754d12a0ee28ed380cfd0d, 7a3eef0fd45f40b783d298ce46aee818; no voters:
I20260811 17:17:10.743542 7898 raft_consensus.cc:2781] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:17:10.743641 7898 raft_consensus.cc:482] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:17:10.743690 7898 raft_consensus.cc:3037] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:17:10.744815 7898 raft_consensus.cc:504] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:10.745198 7898 leader_election.cc:288] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [CANDIDATE]: Term 1 election: Requested vote from peers 7a3eef0fd45f40b783d298ce46aee818 (127.21.54.131:40579), 51791ee80155440d901587802cb71758 (127.21.54.129:45361)
I20260811 17:17:10.745653 7504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ed6815d740948bba31c5976f0b1f26b" candidate_uuid: "29b38e0787754d12a0ee28ed380cfd0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51791ee80155440d901587802cb71758"
I20260811 17:17:10.745636 7818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ed6815d740948bba31c5976f0b1f26b" candidate_uuid: "29b38e0787754d12a0ee28ed380cfd0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a3eef0fd45f40b783d298ce46aee818"
I20260811 17:17:10.745774 7504 raft_consensus.cc:3037] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:17:10.745774 7818 raft_consensus.cc:3037] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:17:10.747125 7818 raft_consensus.cc:2454] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29b38e0787754d12a0ee28ed380cfd0d in term 1.
I20260811 17:17:10.747125 7504 raft_consensus.cc:2454] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29b38e0787754d12a0ee28ed380cfd0d in term 1.
I20260811 17:17:10.747486 7590 leader_election.cc:302] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 29b38e0787754d12a0ee28ed380cfd0d, 51791ee80155440d901587802cb71758, 7a3eef0fd45f40b783d298ce46aee818; no voters:
I20260811 17:17:10.747735 7898 raft_consensus.cc:2781] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:17:10.748021 7898 raft_consensus.cc:686] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [term 1 LEADER]: Becoming Leader. State: Replica: 29b38e0787754d12a0ee28ed380cfd0d, State: Running, Role: LEADER
I20260811 17:17:10.748399 7898 consensus_queue.cc:237] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:10.751819 7340 catalog_manager.cc:5495] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 29b38e0787754d12a0ee28ed380cfd0d (127.21.54.130). New cstate: current_term: 1 leader_uuid: "29b38e0787754d12a0ee28ed380cfd0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } health_report { overall_health: UNKNOWN } } }
I20260811 17:17:10.850343 7888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:10.846483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46680 (local address 127.21.54.130:42201)
0811 17:17:10.846695 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:10.846702 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:10.846720 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:17:10.846797 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:10.846799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:10.846854 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:10.846976 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:17:10.847239 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.847755 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.848825 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:10.849039 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:10.849154 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:10.849488 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:17:10.849928 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:17:10.850069 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:10.850134 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 17:17:10.850192 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 17:17:10.857645 7504 raft_consensus.cc:1261] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Refusing update from remote peer 29b38e0787754d12a0ee28ed380cfd0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:17:10.857645 7818 raft_consensus.cc:1261] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818 [term 1 FOLLOWER]: Refusing update from remote peer 29b38e0787754d12a0ee28ed380cfd0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:17:10.858523 7901 consensus_queue.cc:1035] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:17:10.858801 7898 consensus_queue.cc:1035] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:17:10.869567 7906 mvcc.cc:205] Tried to move back new op lower bound from 7317375511985078272 to 7317375511548174336. Current Snapshot: MvccSnapshot[applied={T|T < 7317375511985078272}]
I20260811 17:17:10.873867 7908 mvcc.cc:205] Tried to move back new op lower bound from 7317375511985078272 to 7317375511548174336. Current Snapshot: MvccSnapshot[applied={T|T < 7317375511985078272}]
I20260811 17:17:10.885596 7340 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:17:10.885880 7340 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:17:10.889696 7340 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448: Sending DeleteTablet for 3 replicas of tablet 2ed6815d740948bba31c5976f0b1f26b
I20260811 17:17:10.890758 7794 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ed6815d740948bba31c5976f0b1f26b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:17:10 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:40248
I20260811 17:17:10.891185 7477 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ed6815d740948bba31c5976f0b1f26b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:17:10 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:48628
I20260811 17:17:10.891502 7639 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ed6815d740948bba31c5976f0b1f26b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:17:10 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:46664
I20260811 17:17:10.892127 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 7408
I20260811 17:17:10.892170 7917 tablet_replica.cc:330] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818: stopping tablet replica
I20260811 17:17:10.894558 7917 raft_consensus.cc:2229] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:17:10.894985 7917 raft_consensus.cc:2258] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:17:10.896728 7917 ts_tablet_manager.cc:1905] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:17:10.899801 7917 ts_tablet_manager.cc:1918] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:17:10.899978 7917 log.cc:1196] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/2ed6815d740948bba31c5976f0b1f26b
I20260811 17:17:10.900388 7917 ts_tablet_manager.cc:1939] T 2ed6815d740948bba31c5976f0b1f26b P 7a3eef0fd45f40b783d298ce46aee818: Deleting consensus metadata
I20260811 17:17:10.900807 7920 tablet_replica.cc:330] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d: stopping tablet replica
I20260811 17:17:10.901365 7920 raft_consensus.cc:2229] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:17:10.901888 7920 raft_consensus.cc:2258] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:17:10.902454 7324 catalog_manager.cc:4841] TS 7a3eef0fd45f40b783d298ce46aee818 (127.21.54.131:40579): tablet 2ed6815d740948bba31c5976f0b1f26b (table test-table [id=a0b13a2b76554ff79dfbbf4f8af08fe1]) successfully deleted
I20260811 17:17:10.906452 7920 ts_tablet_manager.cc:1905] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 17:17:10.909701 7325 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.54.129:45361 (error 108)
I20260811 17:17:10.910751 7920 ts_tablet_manager.cc:1918] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:17:10.910987 7920 log.cc:1196] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/2ed6815d740948bba31c5976f0b1f26b
I20260811 17:17:10.911499 7920 ts_tablet_manager.cc:1939] T 2ed6815d740948bba31c5976f0b1f26b P 29b38e0787754d12a0ee28ed380cfd0d: Deleting consensus metadata
I20260811 17:17:10.911566 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 7567
I20260811 17:17:10.921052 7922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:10.911082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:45361 (local address 127.0.0.1:48630)
0811 17:17:10.920855 (+ 9773us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:10.920937 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.129:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9691,"thread_start_us":69,"threads_started":1}
W20260811 17:17:10.921326 7325 catalog_manager.cc:4584] TS 51791ee80155440d901587802cb71758 (127.21.54.129:45361): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2ed6815d740948bba31c5976f0b1f26b: Network error: Client connection negotiation failed: client connection to 127.21.54.129:45361: connect: Connection refused (error 111)
I20260811 17:17:10.926239 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 7725
I20260811 17:17:10.926502 7922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:10.926137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:42201 (local address 127.0.0.1:46694)
0811 17:17:10.926317 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:10.926390 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.130:42201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 17:17:10.936734 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 7304
I20260811 17:17:30.946460 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:43111
--webserver_interface=127.21.54.190
--webserver_port=39081
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:43111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:17:31.065976 7923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:17:31.066274 7923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:17:31.066316 7923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:17:31.070380 7923 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 17:17:31.070438 7923 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 17:17:31.070456 7923 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 17:17:31.070469 7923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:17:31.070488 7923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:17:31.070506 7923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:17:31.070568 7923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:17:31.070595 7923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:17:31.076090 7923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:43111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:43111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=39081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:17:31.077359 7923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:17:31.078469 7923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:17:31.093847 7930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:31.094336 7929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:31.094376 7932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:17:31.094487 7923 server_base.cc:1034] running on GCE node
I20260811 17:17:31.095197 7923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:17:31.096567 7923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:17:31.097756 7923 hybrid_clock.cc:648] HybridClock initialized: now 1786468651097732 us; error 45 us; skew 500 ppm
Aug 11 17:17:31 dist-test-slave-fd7l krb5kdc[7289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468651, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:17:31.100847 7923 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:17:31.101827 7923 webserver.cc:460] Webserver started at http://127.21.54.190:39081/ using document root <none> and password file <none>
I20260811 17:17:31.102450 7923 fs_manager.cc:359] Metadata directory not provided
I20260811 17:17:31.102562 7923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:17:31.106747 7923 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:17:31.109001 7939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:31.110241 7923 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 17:17:31.110391 7923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "4b9e0859907b48038085e86883d06448"
format_stamp: "Formatted at 2026-08-11 17:17:09 on dist-test-slave-fd7l"
server_key: "3a8c39bdde802ffdda4afa28feb40c1d"
server_key_iv: "fddf60176ba3d5aba2333e91e121c668"
server_key_version: "encryptionkey@0"
I20260811 17:17:31.111016 7923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:17:31.129786 7923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:17:31.133354 7923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:17:31.133615 7923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:17:31.141482 7923 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:43111
I20260811 17:17:31.141803 7999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:43111 every 8 connection(s)
I20260811 17:17:31.142792 7923 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:17:31.145365 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 7923
I20260811 17:17:31.145905 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:45361
--local_ip_for_outbound_sockets=127.21.54.129
--tserver_master_addrs=127.21.54.190:43111
--webserver_port=35165
--webserver_interface=127.21.54.129
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:17:31.148416 8000 sys_catalog.cc:263] Verifying existing consensus state
I20260811 17:17:31.150879 8000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448: Bootstrap starting.
I20260811 17:17:31.170527 8000 log.cc:824] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448: Log is configured to *not* fsync() on all Append() calls
I20260811 17:17:31.180359 8000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:17:31.180866 8000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448: Bootstrap complete.
I20260811 17:17:31.185604 8000 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9e0859907b48038085e86883d06448" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43111 } }
I20260811 17:17:31.186059 8000 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b9e0859907b48038085e86883d06448, State: Initialized, Role: FOLLOWER
I20260811 17:17:31.186604 8000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9e0859907b48038085e86883d06448" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43111 } }
I20260811 17:17:31.186734 8000 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:17:31.186841 8000 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:17:31.186971 8000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 1 FOLLOWER]: Advancing to term 2
I20260811 17:17:31.189252 8000 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9e0859907b48038085e86883d06448" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43111 } }
I20260811 17:17:31.189633 8000 leader_election.cc:302] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b9e0859907b48038085e86883d06448; no voters:
I20260811 17:17:31.190002 8000 leader_election.cc:288] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 17:17:31.190109 8004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 17:17:31.190330 8004 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 2 LEADER]: Becoming Leader. State: Replica: 4b9e0859907b48038085e86883d06448, State: Running, Role: LEADER
I20260811 17:17:31.190688 8004 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9e0859907b48038085e86883d06448" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43111 } }
I20260811 17:17:31.191435 8000 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:17:31.192416 8005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4b9e0859907b48038085e86883d06448" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9e0859907b48038085e86883d06448" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43111 } } }
I20260811 17:17:31.192598 8005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [sys.catalog]: This master's current role is: LEADER
I20260811 17:17:31.193181 8006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b9e0859907b48038085e86883d06448. Latest consensus state: current_term: 2 leader_uuid: "4b9e0859907b48038085e86883d06448" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9e0859907b48038085e86883d06448" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43111 } } }
I20260811 17:17:31.193372 8006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [sys.catalog]: This master's current role is: LEADER
I20260811 17:17:31.193710 8015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:17:31.198927 8015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:17:31.199589 8015 catalog_manager.cc:1259] Loaded cluster ID: bae271c9dfcf402ab74ac3570fde225b
I20260811 17:17:31.199653 8015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:17:31.202225 8015 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:17:31.202926 8015 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448: Loaded TSK: 0
I20260811 17:17:31.216945 8015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448: Generated new TSK 1
I20260811 17:17:31.217307 8015 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 17:17:31.272325 8002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:17:31.272663 8002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:17:31.272758 8002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:17:31.276911 8002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:17:31.276985 8002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:17:31.277047 8002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:17:31.282564 8002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:45361
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=35165
--tserver_master_addrs=127.21.54.190:43111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:17:31.283694 8002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:17:31.284770 8002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:17:31.295192 8031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:31.295238 8030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:31.295192 8035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:17:31.295781 8002 server_base.cc:1034] running on GCE node
I20260811 17:17:31.296260 8002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:17:31.297020 8002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:17:31.298218 8002 hybrid_clock.cc:648] HybridClock initialized: now 1786468651298186 us; error 47 us; skew 500 ppm
Aug 11 17:17:31 dist-test-slave-fd7l krb5kdc[7289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468651, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:17:31.301306 8002 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:17:31.302502 8002 webserver.cc:460] Webserver started at http://127.21.54.129:35165/ using document root <none> and password file <none>
I20260811 17:17:31.303133 8002 fs_manager.cc:359] Metadata directory not provided
I20260811 17:17:31.303249 8002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:17:31.307507 8002 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:17:31.310088 8044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:31.311481 8002 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:17:31.311681 8002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "51791ee80155440d901587802cb71758"
format_stamp: "Formatted at 2026-08-11 17:17:09 on dist-test-slave-fd7l"
server_key: "e1244db4fb83633b37b469f8289ca898"
server_key_iv: "accf92158196876faa4ae1f819fd5f1a"
server_key_version: "encryptionkey@0"
I20260811 17:17:31.312254 8002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:17:31.351330 8002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:17:31.354856 8002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:17:31.355106 8002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:17:31.355829 8002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:17:31.357249 8051 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 17:17:31.361723 8002 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 17:17:31.361815 8002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.001s
I20260811 17:17:31.361919 8002 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 17:17:31.363837 8002 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 17:17:31.363919 8002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 17:17:31.364265 8051 tablet_bootstrap.cc:492] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758: Bootstrap starting.
I20260811 17:17:31.376368 8002 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:45361
I20260811 17:17:31.376463 8169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:45361 every 8 connection(s)
I20260811 17:17:31.377565 8002 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:17:31.385491 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 8002
I20260811 17:17:31.386022 8051 log.cc:824] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758: Log is configured to *not* fsync() on all Append() calls
I20260811 17:17:31.386142 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:42201
--local_ip_for_outbound_sockets=127.21.54.130
--tserver_master_addrs=127.21.54.190:43111
--webserver_port=46137
--webserver_interface=127.21.54.130
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 17:17:31 dist-test-slave-fd7l krb5kdc[7289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468651, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:17:31.395915 8051 tablet_bootstrap.cc:492] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:17:31.396502 8051 tablet_bootstrap.cc:492] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758: Bootstrap complete.
I20260811 17:17:31.398326 8051 ts_tablet_manager.cc:1397] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758: Time spent bootstrapping tablet: real 0.034s user 0.024s sys 0.008s
I20260811 17:17:31.398339 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:31.379965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:43485 (local address 127.21.54.190:43111)
0811 17:17:31.380486 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:31.380547 (+ 61us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:31.380848 (+ 301us) server_negotiation.cc:407] Connection header received
0811 17:17:31.382131 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:31.382162 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:31.382618 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:31.383042 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:17:31.384201 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.385942 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.388011 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.388960 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.389169 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:31.392275 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:17:31.392281 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:17:31.392295 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:17:31.392327 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:17:31.394398 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:31.395018 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:31.395032 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:31.395037 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:31.395240 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:31.395645 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:31.395648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:31.395650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:31.396098 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:17:31.396264 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:31.397047 (+ 783us) server_negotiation.cc:299] Negotiation successful
0811 17:17:31.397357 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":163,"threads_started":1}
I20260811 17:17:31.398350 8173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:31.379862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:43485)
0811 17:17:31.380639 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:31.380699 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 17:17:31.381606 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:17:31.383083 (+ 1477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:17:31.383089 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:17:31.383495 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:17:31.384002 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:31.384018 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.386155 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.386163 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:31.387824 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:31.387842 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.389147 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.389151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:31.389360 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:31.390161 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:17:31.390183 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:17:31.392020 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:17:31.394562 (+ 2542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:17:31.394569 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:17:31.394584 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:17:31.394878 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:17:31.395367 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:17:31.395371 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:17:31.395373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:17:31.395527 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:17:31.396250 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:17:31.396257 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:17:31.396678 (+ 421us) client_negotiation.cc:769] Sending connection context
0811 17:17:31.396918 (+ 240us) client_negotiation.cc:240] Negotiation successful
0811 17:17:31.397319 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":163,"threads_started":1}
I20260811 17:17:31.401424 8170 heartbeater.cc:347] Connected to a master server at 127.21.54.190:43111
I20260811 17:17:31.401713 8170 heartbeater.cc:464] Registering TS with master...
I20260811 17:17:31.402280 8170 heartbeater.cc:511] Master 127.21.54.190:43111 requested a full tablet report, sending...
I20260811 17:17:31.402606 8051 raft_consensus.cc:348] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:31.403321 8051 raft_consensus.cc:729] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51791ee80155440d901587802cb71758, State: Initialized, Role: FOLLOWER
I20260811 17:17:31.403916 8051 consensus_queue.cc:260] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:31.405144 8051 ts_tablet_manager.cc:1428] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 17:17:31.407244 7961 ts_manager.cc:194] Registered new tserver with Master: 51791ee80155440d901587802cb71758 (127.21.54.129:45361)
I20260811 17:17:31.411257 7961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:43485
I20260811 17:17:31.414520 8170 heartbeater.cc:503] Master 127.21.54.190:43111 was elected leader, sending a full tablet report...
Aug 11 17:17:31 dist-test-slave-fd7l krb5kdc[7289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468651, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for kudu/127.21.54.129@KRBTEST.COM
I20260811 17:17:31.422996 8179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:31.410656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39952 (local address 127.21.54.129:45361)
0811 17:17:31.411065 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:31.411077 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:31.411504 (+ 427us) server_negotiation.cc:407] Connection header received
0811 17:17:31.411751 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:31.411757 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:31.411979 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:31.412164 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:17:31.412955 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.414119 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.415408 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.416001 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.416147 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:31.418802 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:17:31.418819 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:17:31.418841 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:17:31.418867 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:17:31.420803 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:31.421385 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:31.421391 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:31.421392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:31.421480 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:31.421802 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:31.421807 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:31.421809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:31.422083 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:17:31.422264 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:31.422654 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 17:17:31.422838 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":123,"threads_started":1}
I20260811 17:17:31.422993 8178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:31.410526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:45361 (local address 127.0.0.1:39952)
0811 17:17:31.411352 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:31.411404 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 17:17:31.411569 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:17:31.412165 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:17:31.412171 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:17:31.412678 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:17:31.412814 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:31.412822 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.414259 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.414262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:31.415246 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:31.415258 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.416148 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.416151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:31.416268 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:31.416931 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:17:31.416946 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:17:31.418642 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:17:31.420932 (+ 2290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:17:31.420945 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:17:31.420952 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:17:31.421237 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:17:31.421589 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:17:31.421592 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:17:31.421595 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:17:31.421697 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:17:31.422233 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:17:31.422240 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:17:31.422389 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 17:17:31.422552 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 17:17:31.422784 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":140,"threads_started":1}
I20260811 17:17:31.424105 8098 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ed6815d740948bba31c5976f0b1f26b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:17:10 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:39952
I20260811 17:17:31.425137 8180 tablet_replica.cc:330] stopping tablet replica
I20260811 17:17:31.425467 8180 raft_consensus.cc:2229] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:17:31.425755 8180 raft_consensus.cc:2258] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:17:31.426862 8180 ts_tablet_manager.cc:1905] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:17:31.430991 8180 ts_tablet_manager.cc:1918] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:17:31.431092 8180 log.cc:1196] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/wal/wals/2ed6815d740948bba31c5976f0b1f26b
I20260811 17:17:31.431409 8180 ts_tablet_manager.cc:1939] T 2ed6815d740948bba31c5976f0b1f26b P 51791ee80155440d901587802cb71758: Deleting consensus metadata
I20260811 17:17:31.433789 7942 catalog_manager.cc:4841] TS 51791ee80155440d901587802cb71758 (127.21.54.129:45361): tablet 2ed6815d740948bba31c5976f0b1f26b (table test-table [id=a0b13a2b76554ff79dfbbf4f8af08fe1]) successfully deleted
W20260811 17:17:31.520709 8175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:17:31.521031 8175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:17:31.521112 8175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:17:31.526418 8175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:17:31.526513 8175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:17:31.526620 8175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:17:31.532653 8175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:42201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=46137
--tserver_master_addrs=127.21.54.190:43111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:17:31.533936 8175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:17:31.535060 8175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:17:31.545557 8175 server_base.cc:1034] running on GCE node
W20260811 17:17:31.545511 8186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:31.545500 8187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:31.545497 8189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:17:31.546288 8175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:17:31.547020 8175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:17:31.548223 8175 hybrid_clock.cc:648] HybridClock initialized: now 1786468651548196 us; error 39 us; skew 500 ppm
Aug 11 17:17:31 dist-test-slave-fd7l krb5kdc[7289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468651, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:17:31.551209 8175 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:17:31.552099 8175 webserver.cc:460] Webserver started at http://127.21.54.130:46137/ using document root <none> and password file <none>
I20260811 17:17:31.552737 8175 fs_manager.cc:359] Metadata directory not provided
I20260811 17:17:31.552816 8175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:17:31.556908 8175 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260811 17:17:31.559193 8196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:31.560812 8175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:17:31.560946 8175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "29b38e0787754d12a0ee28ed380cfd0d"
format_stamp: "Formatted at 2026-08-11 17:17:10 on dist-test-slave-fd7l"
server_key: "300f8ee1db0729a9b5954a69410675a4"
server_key_iv: "4f07085aee45e845441f1dfb6a66d82a"
server_key_version: "encryptionkey@0"
I20260811 17:17:31.561633 8175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:17:31.590102 8175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:17:31.594305 8175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:17:31.594612 8175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:17:31.595405 8175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:17:31.596390 8175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:17:31.596443 8175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:31.596571 8175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:17:31.596606 8175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:31.606899 8175 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:42201
I20260811 17:17:31.606918 8324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:42201 every 8 connection(s)
I20260811 17:17:31.608078 8175 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:17:31.609790 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 8175
I20260811 17:17:31.610378 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:40579
--local_ip_for_outbound_sockets=127.21.54.131
--tserver_master_addrs=127.21.54.190:43111
--webserver_port=46243
--webserver_interface=127.21.54.131
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 17:17:31 dist-test-slave-fd7l krb5kdc[7289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468651, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:17:31.624534 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:31.610162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:36913 (local address 127.21.54.190:43111)
0811 17:17:31.610341 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:31.610347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:31.611454 (+ 1107us) server_negotiation.cc:407] Connection header received
0811 17:17:31.613275 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:31.613279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:31.613353 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:31.613475 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:17:31.615482 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.616203 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.617655 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.617901 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.618024 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:31.620773 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:17:31.620776 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:17:31.620778 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:17:31.620806 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:17:31.622359 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:31.622908 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:31.622911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:31.622912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:31.622989 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:31.623347 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:31.623350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:31.623352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:31.623541 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:17:31.623654 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:31.624181 (+ 527us) server_negotiation.cc:299] Negotiation successful
0811 17:17:31.624324 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 17:17:31.625402 8328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:31.610736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:36913)
0811 17:17:31.611266 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:31.611324 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 17:17:31.613010 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:17:31.613738 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:17:31.613752 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:17:31.614349 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:17:31.615220 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:31.615245 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.616372 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.616375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:31.617518 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:31.617528 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.618077 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.618081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:31.618264 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:31.618976 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:17:31.618994 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:17:31.620487 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:17:31.622488 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:17:31.622494 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:17:31.622512 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:17:31.622787 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:17:31.623084 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:17:31.623088 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:17:31.623090 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:17:31.623241 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:17:31.623655 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:17:31.623662 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:17:31.623922 (+ 260us) client_negotiation.cc:769] Sending connection context
0811 17:17:31.624117 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 17:17:31.624463 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":164,"threads_started":1}
I20260811 17:17:31.626682 8325 heartbeater.cc:347] Connected to a master server at 127.21.54.190:43111
I20260811 17:17:31.626960 8325 heartbeater.cc:464] Registering TS with master...
I20260811 17:17:31.627499 8325 heartbeater.cc:511] Master 127.21.54.190:43111 requested a full tablet report, sending...
I20260811 17:17:31.628700 7961 ts_manager.cc:194] Registered new tserver with Master: 29b38e0787754d12a0ee28ed380cfd0d (127.21.54.130:42201)
I20260811 17:17:31.629567 7961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:36913
W20260811 17:17:31.736340 8329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:17:31.736636 8329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:17:31.736681 8329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:17:31.740754 8329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:17:31.740816 8329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:17:31.740875 8329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:17:31.746124 8329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:40579
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=46243
--tserver_master_addrs=127.21.54.190:43111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:17:31.747166 8329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:17:31.748346 8329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:17:31.758947 8337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:31.758941 8335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:31.759021 8334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:17:31.759315 8329 server_base.cc:1034] running on GCE node
I20260811 17:17:31.759825 8329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:17:31.760618 8329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:17:31.761821 8329 hybrid_clock.cc:648] HybridClock initialized: now 1786468651761768 us; error 58 us; skew 500 ppm
Aug 11 17:17:31 dist-test-slave-fd7l krb5kdc[7289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468651, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:17:31.764912 8329 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:17:31.765980 8329 webserver.cc:460] Webserver started at http://127.21.54.131:46243/ using document root <none> and password file <none>
I20260811 17:17:31.766772 8329 fs_manager.cc:359] Metadata directory not provided
I20260811 17:17:31.766897 8329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:17:31.771466 8329 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:17:31.773914 8345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:31.775435 8329 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:17:31.775606 8329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "7a3eef0fd45f40b783d298ce46aee818"
format_stamp: "Formatted at 2026-08-11 17:17:10 on dist-test-slave-fd7l"
server_key: "7005e6ad7aba23ea0f06438f031d5f2e"
server_key_iv: "1df7625b7e605e91be196d54ab2eef42"
server_key_version: "encryptionkey@0"
I20260811 17:17:31.776233 8329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:17:31.800421 8329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:17:31.804176 8329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:17:31.804426 8329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:17:31.805213 8329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:17:31.806162 8329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:17:31.806244 8329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:31.806324 8329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:17:31.806381 8329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:31.817340 8329 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:40579
I20260811 17:17:31.817364 8474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:40579 every 8 connection(s)
I20260811 17:17:31.818567 8329 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:17:31.820241 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 8329
Aug 11 17:17:31 dist-test-slave-fd7l krb5kdc[7289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468651, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:17:31.830468 8478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:31.821411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52506 (local address 127.21.54.190:43111)
0811 17:17:31.821715 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:31.821719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:31.821732 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:17:31.821776 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:31.821779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:31.821819 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:31.821930 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:17:31.822838 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.823474 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.824558 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.824845 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.824962 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:31.826537 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:17:31.826541 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:17:31.826543 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:17:31.826572 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:17:31.828373 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:31.829004 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:31.829006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:31.829008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:31.829118 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:31.829443 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:31.829446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:31.829448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:31.829681 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:17:31.829910 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:31.830145 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 17:17:31.830274 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":103,"threads_started":1}
I20260811 17:17:31.834721 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:31.820821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:36937 (local address 127.21.54.190:43111)
0811 17:17:31.820975 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:31.820980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:31.822197 (+ 1217us) server_negotiation.cc:407] Connection header received
0811 17:17:31.823679 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:31.823683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:31.823739 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:31.823817 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:17:31.825161 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.825974 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.827290 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.827551 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.827676 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:31.830803 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:17:31.830807 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:17:31.830810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:17:31.830838 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:17:31.832504 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:31.833155 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:31.833158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:31.833160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:31.833249 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:31.833572 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:31.833574 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:31.833576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:31.833771 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:17:31.833863 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:31.834479 (+ 616us) server_negotiation.cc:299] Negotiation successful
0811 17:17:31.834592 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 17:17:31.835644 8479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:31.821422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:36937)
0811 17:17:31.822012 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:31.822069 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 17:17:31.823488 (+ 1419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:17:31.823983 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:17:31.823991 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:17:31.824366 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:17:31.824997 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:31.825013 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.826124 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.826128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:31.827145 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:31.827157 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.827730 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.827735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:31.827932 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:31.828595 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:17:31.828617 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:17:31.830618 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:17:31.832673 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:17:31.832681 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:17:31.832702 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:17:31.833013 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:17:31.833340 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:17:31.833342 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:17:31.833343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:17:31.833476 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:17:31.833916 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:17:31.833922 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:17:31.834213 (+ 291us) client_negotiation.cc:769] Sending connection context
0811 17:17:31.834400 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 17:17:31.834688 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":171,"threads_started":1}
I20260811 17:17:31.837136 8475 heartbeater.cc:347] Connected to a master server at 127.21.54.190:43111
I20260811 17:17:31.837393 8475 heartbeater.cc:464] Registering TS with master...
I20260811 17:17:31.837903 8475 heartbeater.cc:511] Master 127.21.54.190:43111 requested a full tablet report, sending...
I20260811 17:17:31.838990 7961 ts_manager.cc:194] Registered new tserver with Master: 7a3eef0fd45f40b783d298ce46aee818 (127.21.54.131:40579)
I20260811 17:17:31.839624 7961 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:36937
I20260811 17:17:31.844794 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:17:31.849014 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:31.845509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52508 (local address 127.21.54.190:43111)
0811 17:17:31.845690 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:31.845697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:31.845845 (+ 148us) server_negotiation.cc:407] Connection header received
0811 17:17:31.845989 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:31.845993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:31.846110 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:31.846204 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:17:31.846434 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.846993 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.847887 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.848105 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.848185 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:31.848459 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:17:31.848662 (+ 203us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 17:17:31.848886 (+ 224us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52508: token expired
Metrics: {"server-negotiator.queue_time_us":93}
W20260811 17:17:31.849098 8172 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52508: token expired
W20260811 17:17:31.849478 7876 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.21.54.190:43111)
I20260811 17:17:31.856813 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:31.849870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52512 (local address 127.21.54.190:43111)
0811 17:17:31.849987 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:31.849997 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:31.850011 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:17:31.850096 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:31.850098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:31.850157 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:31.850225 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:17:31.850852 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.851338 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.852139 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.852340 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.852419 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:31.853803 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:17:31.853806 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:17:31.853808 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:17:31.853831 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:17:31.855343 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:31.855731 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:31.855733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:31.855734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:31.855810 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:31.856039 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:31.856042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:31.856043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:31.856224 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:17:31.856328 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:31.856581 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 17:17:31.856683 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 17:17:31.859510 7961 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:17:31.863152 7961 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:17:31.871896 8098 tablet_service.cc:1467] Processing CreateTablet for tablet 472a57249ee94ea289f1e5c151e53b54 (DEFAULT_TABLE table=test-table [id=2707ddb92a5b400b928b6e47a010cc5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:17:31.872663 8098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 472a57249ee94ea289f1e5c151e53b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:17:31.875908 8051 tablet_bootstrap.cc:492] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758: Bootstrap starting.
I20260811 17:17:31.876868 8051 tablet_bootstrap.cc:654] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758: Neither blocks nor log segments found. Creating new log.
I20260811 17:17:31.877548 8489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:31.871694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33884 (local address 127.21.54.130:42201)
0811 17:17:31.872031 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:31.872041 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:31.872083 (+ 42us) server_negotiation.cc:407] Connection header received
0811 17:17:31.872189 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:31.872195 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:31.872452 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:31.872651 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:17:31.873366 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.874131 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.875800 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.876750 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.876874 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:31.877072 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:31.877260 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 17:17:31.877374 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":134,"threads_started":1}
I20260811 17:17:31.877570 8178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:31.870791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:40579 (local address 127.0.0.1:38420)
0811 17:17:31.870916 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:31.870931 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:17:31.871035 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:17:31.872869 (+ 1834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:17:31.872873 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:17:31.872905 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:17:31.873112 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:31.873121 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.874531 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.874534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:31.875714 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:31.875726 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.877129 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.877132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:31.877247 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:31.877283 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 17:17:31.877337 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 17:17:31.877413 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":43,"spinlock_wait_cycles":3840}
I20260811 17:17:31.878157 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:31.871230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:42201 (local address 127.0.0.1:33884)
0811 17:17:31.871504 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:31.871516 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 17:17:31.871630 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:17:31.872867 (+ 1237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:17:31.872871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:17:31.872908 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:17:31.873255 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:31.873266 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.874253 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.874259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:31.875634 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:31.875647 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.876914 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.876920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:31.877025 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:31.877100 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 17:17:31.877222 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 17:17:31.877343 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":82,"spinlock_wait_cycles":5888,"thread_start_us":138,"threads_started":1}
I20260811 17:17:31.878665 8487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:31.871247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38420 (local address 127.21.54.131:40579)
0811 17:17:31.872332 (+ 1085us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:31.872359 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:31.872386 (+ 27us) server_negotiation.cc:407] Connection header received
0811 17:17:31.872461 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:31.872466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:31.872698 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:31.873130 (+ 432us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:17:31.873330 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.874410 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.875848 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:31.876874 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:31.878152 (+ 1278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:31.878337 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:31.878396 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 17:17:31.878501 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":823,"thread_start_us":149,"threads_started":1}
I20260811 17:17:31.878728 8051 tablet_bootstrap.cc:492] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758: No bootstrap required, opened a new log
I20260811 17:17:31.878822 8051 ts_tablet_manager.cc:1397] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:17:31.879357 8051 raft_consensus.cc:348] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:31.879520 8051 raft_consensus.cc:374] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:17:31.879592 8051 raft_consensus.cc:729] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51791ee80155440d901587802cb71758, State: Initialized, Role: FOLLOWER
I20260811 17:17:31.879761 8051 consensus_queue.cc:260] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:31.880061 8051 ts_tablet_manager.cc:1428] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 17:17:31.881762 8401 tablet_service.cc:1467] Processing CreateTablet for tablet 472a57249ee94ea289f1e5c151e53b54 (DEFAULT_TABLE table=test-table [id=2707ddb92a5b400b928b6e47a010cc5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:17:31.881762 8250 tablet_service.cc:1467] Processing CreateTablet for tablet 472a57249ee94ea289f1e5c151e53b54 (DEFAULT_TABLE table=test-table [id=2707ddb92a5b400b928b6e47a010cc5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:17:31.882961 8250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 472a57249ee94ea289f1e5c151e53b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:17:31.882957 8401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 472a57249ee94ea289f1e5c151e53b54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:17:31.891742 8493 tablet_bootstrap.cc:492] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d: Bootstrap starting.
I20260811 17:17:31.891742 8492 tablet_bootstrap.cc:492] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818: Bootstrap starting.
I20260811 17:17:31.894397 8492 tablet_bootstrap.cc:654] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818: Neither blocks nor log segments found. Creating new log.
I20260811 17:17:31.894979 8493 tablet_bootstrap.cc:654] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d: Neither blocks nor log segments found. Creating new log.
I20260811 17:17:31.895857 8492 log.cc:824] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818: Log is configured to *not* fsync() on all Append() calls
I20260811 17:17:31.895857 8493 log.cc:824] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d: Log is configured to *not* fsync() on all Append() calls
I20260811 17:17:31.898023 8493 tablet_bootstrap.cc:492] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d: No bootstrap required, opened a new log
I20260811 17:17:31.898068 8492 tablet_bootstrap.cc:492] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818: No bootstrap required, opened a new log
I20260811 17:17:31.898252 8493 ts_tablet_manager.cc:1397] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:17:31.898275 8492 ts_tablet_manager.cc:1397] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:17:31.901624 8493 raft_consensus.cc:348] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:31.901875 8493 raft_consensus.cc:374] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:17:31.901705 8492 raft_consensus.cc:348] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:31.901934 8493 raft_consensus.cc:729] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29b38e0787754d12a0ee28ed380cfd0d, State: Initialized, Role: FOLLOWER
I20260811 17:17:31.901969 8492 raft_consensus.cc:374] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:17:31.902060 8492 raft_consensus.cc:729] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a3eef0fd45f40b783d298ce46aee818, State: Initialized, Role: FOLLOWER
I20260811 17:17:31.902473 8493 consensus_queue.cc:260] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:31.902487 8492 consensus_queue.cc:260] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:31.903231 8475 heartbeater.cc:503] Master 127.21.54.190:43111 was elected leader, sending a full tablet report...
I20260811 17:17:31.903545 8493 ts_tablet_manager.cc:1428] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260811 17:17:31.903569 8492 ts_tablet_manager.cc:1428] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 17:17:31.903909 8325 heartbeater.cc:503] Master 127.21.54.190:43111 was elected leader, sending a full tablet report...
W20260811 17:17:32.069693 8476 tablet.cc:2369] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:17:32.109618 8327 tablet.cc:2369] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:17:32.129635 8171 tablet.cc:2369] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:17:32.274009 8496 raft_consensus.cc:482] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:17:32.274268 8496 raft_consensus.cc:504] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:32.275563 8496 leader_election.cc:288] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a3eef0fd45f40b783d298ce46aee818 (127.21.54.131:40579), 51791ee80155440d901587802cb71758 (127.21.54.129:45361)
I20260811 17:17:32.280074 8499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:32.275891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:40579 (local address 127.21.54.130:60279)
0811 17:17:32.276277 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:32.276295 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:17:32.276434 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:17:32.276790 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:17:32.276794 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:17:32.276842 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:17:32.277039 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:32.277051 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:32.277828 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:32.277831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:32.278816 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:32.278826 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:32.279524 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:32.279529 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:32.279641 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:32.279685 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 17:17:32.279772 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 17:17:32.279858 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":33,"spinlock_wait_cycles":3712,"thread_start_us":67,"threads_started":1}
I20260811 17:17:32.280119 8487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:32.275993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:60279 (local address 127.21.54.131:40579)
0811 17:17:32.276228 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:32.276236 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:32.276402 (+ 166us) server_negotiation.cc:407] Connection header received
0811 17:17:32.276622 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:32.276625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:32.276669 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:32.276763 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:17:32.277168 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:32.277701 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:32.278989 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:32.279377 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:32.279478 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:32.279589 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:32.279823 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 17:17:32.279920 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260811 17:17:32.280438 8498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:32.275883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:45361 (local address 127.21.54.130:37753)
0811 17:17:32.276200 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:32.276217 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:17:32.276351 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:17:32.276787 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:17:32.276790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:17:32.276839 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:17:32.277031 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:32.277037 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:32.277959 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:32.277962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:32.278810 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:32.278821 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:32.279820 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:32.279823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:32.279925 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:32.279947 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 17:17:32.280032 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 17:17:32.280207 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":45,"spinlock_wait_cycles":44032,"thread_start_us":84,"threads_started":1}
I20260811 17:17:32.280586 8500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:32.276052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:37753 (local address 127.21.54.129:45361)
0811 17:17:32.276352 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:32.276358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:32.276378 (+ 20us) server_negotiation.cc:407] Connection header received
0811 17:17:32.276552 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:32.276556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:32.276639 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:32.276778 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:17:32.277225 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:32.277838 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:32.278961 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:32.279633 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:32.279790 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:32.279965 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:32.280061 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 17:17:32.280169 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":107,"threads_started":1}
I20260811 17:17:32.281088 8422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "472a57249ee94ea289f1e5c151e53b54" candidate_uuid: "29b38e0787754d12a0ee28ed380cfd0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a3eef0fd45f40b783d298ce46aee818" is_pre_election: true
I20260811 17:17:32.281170 8120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "472a57249ee94ea289f1e5c151e53b54" candidate_uuid: "29b38e0787754d12a0ee28ed380cfd0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51791ee80155440d901587802cb71758" is_pre_election: true
I20260811 17:17:32.281473 8422 raft_consensus.cc:2454] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29b38e0787754d12a0ee28ed380cfd0d in term 0.
I20260811 17:17:32.281526 8120 raft_consensus.cc:2454] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29b38e0787754d12a0ee28ed380cfd0d in term 0.
I20260811 17:17:32.282044 8198 leader_election.cc:302] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29b38e0787754d12a0ee28ed380cfd0d, 7a3eef0fd45f40b783d298ce46aee818; no voters:
I20260811 17:17:32.282379 8496 raft_consensus.cc:2781] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:17:32.282491 8496 raft_consensus.cc:482] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:17:32.282563 8496 raft_consensus.cc:3037] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:17:32.283759 8496 raft_consensus.cc:504] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:32.284200 8496 leader_election.cc:288] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [CANDIDATE]: Term 1 election: Requested vote from peers 7a3eef0fd45f40b783d298ce46aee818 (127.21.54.131:40579), 51791ee80155440d901587802cb71758 (127.21.54.129:45361)
I20260811 17:17:32.284678 8120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "472a57249ee94ea289f1e5c151e53b54" candidate_uuid: "29b38e0787754d12a0ee28ed380cfd0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51791ee80155440d901587802cb71758"
I20260811 17:17:32.284754 8422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "472a57249ee94ea289f1e5c151e53b54" candidate_uuid: "29b38e0787754d12a0ee28ed380cfd0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a3eef0fd45f40b783d298ce46aee818"
I20260811 17:17:32.284897 8422 raft_consensus.cc:3037] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:17:32.284884 8120 raft_consensus.cc:3037] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:17:32.286049 8120 raft_consensus.cc:2454] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29b38e0787754d12a0ee28ed380cfd0d in term 1.
I20260811 17:17:32.286063 8422 raft_consensus.cc:2454] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29b38e0787754d12a0ee28ed380cfd0d in term 1.
I20260811 17:17:32.286394 8200 leader_election.cc:302] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29b38e0787754d12a0ee28ed380cfd0d, 51791ee80155440d901587802cb71758; no voters:
I20260811 17:17:32.286679 8496 raft_consensus.cc:2781] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:17:32.286906 8496 raft_consensus.cc:686] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [term 1 LEADER]: Becoming Leader. State: Replica: 29b38e0787754d12a0ee28ed380cfd0d, State: Running, Role: LEADER
I20260811 17:17:32.287318 8496 consensus_queue.cc:237] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:32.290875 7960 catalog_manager.cc:5495] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 29b38e0787754d12a0ee28ed380cfd0d (127.21.54.130). New cstate: current_term: 1 leader_uuid: "29b38e0787754d12a0ee28ed380cfd0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } health_report { overall_health: UNKNOWN } } }
I20260811 17:17:32.339123 8489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:32.335064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33896 (local address 127.21.54.130:42201)
0811 17:17:32.335201 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:32.335207 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:32.335267 (+ 60us) server_negotiation.cc:407] Connection header received
0811 17:17:32.335481 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:32.335484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:32.335596 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:32.335726 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:17:32.335974 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:32.336670 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:32.337530 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:32.337761 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:32.337872 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:32.338234 (+ 362us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:17:32.338704 (+ 470us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:17:32.338850 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:32.338889 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 17:17:32.338963 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:17:32.345696 8120 raft_consensus.cc:1261] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Refusing update from remote peer 29b38e0787754d12a0ee28ed380cfd0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:17:32.345700 8422 raft_consensus.cc:1261] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818 [term 1 FOLLOWER]: Refusing update from remote peer 29b38e0787754d12a0ee28ed380cfd0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:17:32.346395 8501 consensus_queue.cc:1035] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:17:32.346598 8496 consensus_queue.cc:1035] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:17:32.353412 8503 mvcc.cc:205] Tried to move back new op lower bound from 7317375600000155648 to 7317375599771693056. Current Snapshot: MvccSnapshot[applied={T|T < 7317375600000155648}]
I20260811 17:17:32.363672 7960 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:17:32.363879 7960 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:17:32.366564 7960 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448: Sending DeleteTablet for 3 replicas of tablet 472a57249ee94ea289f1e5c151e53b54
I20260811 17:17:32.367435 8098 tablet_service.cc:1514] Processing DeleteTablet for tablet 472a57249ee94ea289f1e5c151e53b54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:17:32 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:39952
I20260811 17:17:32.367455 8401 tablet_service.cc:1514] Processing DeleteTablet for tablet 472a57249ee94ea289f1e5c151e53b54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:17:32 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:38420
I20260811 17:17:32.367504 8250 tablet_service.cc:1514] Processing DeleteTablet for tablet 472a57249ee94ea289f1e5c151e53b54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:17:32 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:33884
I20260811 17:17:32.368170 8516 tablet_replica.cc:330] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818: stopping tablet replica
I20260811 17:17:32.368218 8515 tablet_replica.cc:330] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d: stopping tablet replica
I20260811 17:17:32.368444 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 8002
I20260811 17:17:32.368609 8515 raft_consensus.cc:2229] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:17:32.368666 8516 raft_consensus.cc:2229] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:17:32.369128 8516 raft_consensus.cc:2258] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:17:32.369956 8515 raft_consensus.cc:2258] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:17:32.371788 8516 ts_tablet_manager.cc:1905] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:17:32.374600 8516 ts_tablet_manager.cc:1918] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:17:32.374715 8516 log.cc:1196] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/472a57249ee94ea289f1e5c151e53b54
I20260811 17:17:32.375075 8516 ts_tablet_manager.cc:1939] T 472a57249ee94ea289f1e5c151e53b54 P 7a3eef0fd45f40b783d298ce46aee818: Deleting consensus metadata
I20260811 17:17:32.375312 8515 ts_tablet_manager.cc:1905] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:17:32.377180 7941 catalog_manager.cc:4841] TS 7a3eef0fd45f40b783d298ce46aee818 (127.21.54.131:40579): tablet 472a57249ee94ea289f1e5c151e53b54 (table test-table [id=2707ddb92a5b400b928b6e47a010cc5a]) successfully deleted
I20260811 17:17:32.379069 8515 ts_tablet_manager.cc:1918] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:17:32.379232 8515 log.cc:1196] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/472a57249ee94ea289f1e5c151e53b54
I20260811 17:17:32.379626 8515 ts_tablet_manager.cc:1939] T 472a57249ee94ea289f1e5c151e53b54 P 29b38e0787754d12a0ee28ed380cfd0d: Deleting consensus metadata
I20260811 17:17:32.381156 7940 catalog_manager.cc:4841] TS 29b38e0787754d12a0ee28ed380cfd0d (127.21.54.130:42201): tablet 472a57249ee94ea289f1e5c151e53b54 (table test-table [id=2707ddb92a5b400b928b6e47a010cc5a]) successfully deleted
W20260811 17:17:32.381954 7942 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.54.129:45361 (error 108)
I20260811 17:17:32.382664 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 8175
I20260811 17:17:32.383556 8518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:32.382780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:45361 (local address 127.0.0.1:39964)
0811 17:17:32.383335 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:32.383422 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.129:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":62,"threads_started":1}
W20260811 17:17:32.384951 7942 catalog_manager.cc:4584] TS 51791ee80155440d901587802cb71758 (127.21.54.129:45361): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 472a57249ee94ea289f1e5c151e53b54: Network error: Client connection negotiation failed: client connection to 127.21.54.129:45361: connect: Connection refused (error 111)
I20260811 17:17:32.393157 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 8329
I20260811 17:17:32.403048 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 7923
I20260811 17:17:52.414295 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:43111
--webserver_interface=127.21.54.190
--webserver_port=39081
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:43111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:17:52.539108 8519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:17:52.539426 8519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:17:52.539470 8519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:17:52.543879 8519 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 17:17:52.543947 8519 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 17:17:52.543965 8519 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 17:17:52.543979 8519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:17:52.544000 8519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:17:52.544020 8519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:17:52.544078 8519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:17:52.544106 8519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:17:52.549877 8519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:43111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:43111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=39081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:17:52.551360 8519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:17:52.552598 8519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:17:52.562176 8525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:52.562176 8527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:52.562177 8524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:17:52.562580 8519 server_base.cc:1034] running on GCE node
I20260811 17:17:52.563292 8519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:17:52.564735 8519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:17:52.565953 8519 hybrid_clock.cc:648] HybridClock initialized: now 1786468672565931 us; error 37 us; skew 500 ppm
Aug 11 17:17:52 dist-test-slave-fd7l krb5kdc[7289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468672, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:17:52.569171 8519 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:17:52.570247 8519 webserver.cc:460] Webserver started at http://127.21.54.190:39081/ using document root <none> and password file <none>
I20260811 17:17:52.570873 8519 fs_manager.cc:359] Metadata directory not provided
I20260811 17:17:52.570972 8519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:17:52.575361 8519 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:17:52.578106 8535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:52.579835 8519 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 17:17:52.580049 8519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "4b9e0859907b48038085e86883d06448"
format_stamp: "Formatted at 2026-08-11 17:17:09 on dist-test-slave-fd7l"
server_key: "3a8c39bdde802ffdda4afa28feb40c1d"
server_key_iv: "fddf60176ba3d5aba2333e91e121c668"
server_key_version: "encryptionkey@0"
I20260811 17:17:52.580796 8519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:17:52.597472 8519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:17:52.601404 8519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:17:52.601656 8519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:17:52.611435 8519 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:43111
I20260811 17:17:52.611742 8599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:43111 every 8 connection(s)
I20260811 17:17:52.612824 8519 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:17:52.613346 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 8519
I20260811 17:17:52.613873 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:45361
--local_ip_for_outbound_sockets=127.21.54.129
--tserver_master_addrs=127.21.54.190:43111
--webserver_port=35165
--webserver_interface=127.21.54.129
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:17:52.617678 8600 sys_catalog.cc:263] Verifying existing consensus state
I20260811 17:17:52.619500 8600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448: Bootstrap starting.
I20260811 17:17:52.638001 8600 log.cc:824] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448: Log is configured to *not* fsync() on all Append() calls
I20260811 17:17:52.651219 8600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:17:52.651706 8600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448: Bootstrap complete.
I20260811 17:17:52.656000 8600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9e0859907b48038085e86883d06448" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43111 } }
I20260811 17:17:52.656405 8600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b9e0859907b48038085e86883d06448, State: Initialized, Role: FOLLOWER
I20260811 17:17:52.657004 8600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9e0859907b48038085e86883d06448" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43111 } }
I20260811 17:17:52.657135 8600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:17:52.657231 8600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:17:52.657363 8600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 2 FOLLOWER]: Advancing to term 3
I20260811 17:17:52.659358 8600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9e0859907b48038085e86883d06448" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43111 } }
I20260811 17:17:52.659775 8600 leader_election.cc:302] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b9e0859907b48038085e86883d06448; no voters:
I20260811 17:17:52.660158 8600 leader_election.cc:288] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 17:17:52.660259 8605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 17:17:52.660493 8605 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [term 3 LEADER]: Becoming Leader. State: Replica: 4b9e0859907b48038085e86883d06448, State: Running, Role: LEADER
I20260811 17:17:52.660902 8605 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9e0859907b48038085e86883d06448" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43111 } }
I20260811 17:17:52.661550 8600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:17:52.661927 8605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b9e0859907b48038085e86883d06448. Latest consensus state: current_term: 3 leader_uuid: "4b9e0859907b48038085e86883d06448" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9e0859907b48038085e86883d06448" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43111 } } }
I20260811 17:17:52.662046 8605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [sys.catalog]: This master's current role is: LEADER
I20260811 17:17:52.662514 8611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:17:52.662523 8606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4b9e0859907b48038085e86883d06448" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9e0859907b48038085e86883d06448" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 43111 } } }
I20260811 17:17:52.662776 8606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448 [sys.catalog]: This master's current role is: LEADER
I20260811 17:17:52.667999 8611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:17:52.668802 8611 catalog_manager.cc:1259] Loaded cluster ID: bae271c9dfcf402ab74ac3570fde225b
I20260811 17:17:52.668889 8611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:17:52.670751 8611 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:17:52.671411 8611 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448: Loaded TSK: 0
I20260811 17:17:52.671507 8611 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4b9e0859907b48038085e86883d06448: Loaded TSK: 1
F20260811 17:17:52.672133 8611 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f4899e219ad google::LogMessage::Fail() at ??:0
@ 0x7f4899e2488a google::LogMessage::SendToLog() at ??:0
@ 0x7f4899e214de google::LogMessage::Flush() at ??:0
@ 0x7f4899e22a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f489cae518d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f489caddf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f48a47867bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f48a476099c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f48a4761178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f48a47a2df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f48a47c182b std::function<>::operator()() at ??:0
@ 0x7f48a4760b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f48a4761f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f48a475e579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f48a47a2640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f489c0733d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f489c073c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f489c0753f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f489c064512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f489d6a86db start_thread at ??:0
@ 0x7f489891f61f clone at ??:0
W20260811 17:17:52.759155 8602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:17:52.759411 8602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:17:52.759466 8602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:17:52.764572 8602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:17:52.764631 8602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:17:52.764695 8602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:17:52.771075 8602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:45361
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=35165
--tserver_master_addrs=127.21.54.190:43111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:17:52.772305 8602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:17:52.773397 8602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:17:52.783942 8657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:17:52.784212 8602 server_base.cc:1034] running on GCE node
W20260811 17:17:52.783989 8654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:52.783968 8655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:17:52.784930 8602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:17:52.785658 8602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:17:52.786865 8602 hybrid_clock.cc:648] HybridClock initialized: now 1786468672786839 us; error 45 us; skew 500 ppm
Aug 11 17:17:52 dist-test-slave-fd7l krb5kdc[7289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468672, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:17:52.790115 8602 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:17:52.791193 8602 webserver.cc:460] Webserver started at http://127.21.54.129:35165/ using document root <none> and password file <none>
I20260811 17:17:52.791889 8602 fs_manager.cc:359] Metadata directory not provided
I20260811 17:17:52.791987 8602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:17:52.796296 8602 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:17:52.798764 8664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:52.800205 8602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:17:52.800334 8602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "51791ee80155440d901587802cb71758"
format_stamp: "Formatted at 2026-08-11 17:17:09 on dist-test-slave-fd7l"
server_key: "e1244db4fb83633b37b469f8289ca898"
server_key_iv: "accf92158196876faa4ae1f819fd5f1a"
server_key_version: "encryptionkey@0"
I20260811 17:17:52.801003 8602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:17:52.818861 8602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:17:52.822485 8602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:17:52.822715 8602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:17:52.823380 8602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:17:52.824612 8671 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 17:17:52.830611 8602 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 17:17:52.830696 8602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260811 17:17:52.830785 8602 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 17:17:52.832803 8602 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 17:17:52.832875 8602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260811 17:17:52.833180 8671 tablet_bootstrap.cc:492] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758: Bootstrap starting.
I20260811 17:17:52.844086 8602 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:45361
I20260811 17:17:52.844801 8794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:45361 every 8 connection(s)
I20260811 17:17:52.845309 8602 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:17:52.847237 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 8602
I20260811 17:17:52.847772 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:42201
--local_ip_for_outbound_sockets=127.21.54.130
--tserver_master_addrs=127.21.54.190:43111
--webserver_port=46137
--webserver_interface=127.21.54.130
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:17:52.857388 8671 log.cc:824] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758: Log is configured to *not* fsync() on all Append() calls
I20260811 17:17:52.865080 8671 tablet_bootstrap.cc:492] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:17:52.865520 8671 tablet_bootstrap.cc:492] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758: Bootstrap complete.
I20260811 17:17:52.866659 8671 ts_tablet_manager.cc:1397] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758: Time spent bootstrapping tablet: real 0.034s user 0.018s sys 0.011s
I20260811 17:17:52.869237 8671 raft_consensus.cc:348] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:52.869702 8671 raft_consensus.cc:729] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51791ee80155440d901587802cb71758, State: Initialized, Role: FOLLOWER
I20260811 17:17:52.870223 8671 consensus_queue.cc:260] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:52.871255 8671 ts_tablet_manager.cc:1428] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260811 17:17:52.944170 8797 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:17:52.847594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:44917)
0811 17:17:52.849047 (+ 1453us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:52.849115 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 17:17:52.850215 (+ 1100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:17:52.943167 (+ 92952us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1189,"thread_start_us":165,"threads_started":1}
W20260811 17:17:52.944875 8795 heartbeater.cc:644] Failed to heartbeat to 127.21.54.190:43111 (0 consecutive failures): Network error: Failed to ping master at 127.21.54.190:43111: Client connection negotiation failed: client connection to 127.21.54.190:43111: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 17:17:52.945758 8797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:52.945433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:58903)
0811 17:17:52.945620 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:52.945698 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 17:17:52.946487 8797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:52.946274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:42365)
0811 17:17:52.946389 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:52.946433 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 17:17:52.946712 8795 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 17:17:52.989704 8798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:17:52.989998 8798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:17:52.990044 8798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:17:52.994354 8798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:17:52.994432 8798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:17:52.994501 8798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:17:53.000098 8798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:42201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=46137
--tserver_master_addrs=127.21.54.190:43111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:17:53.001333 8798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:17:53.002377 8798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:17:53.012451 8805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:53.012480 8806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:17:53.012552 8798 server_base.cc:1034] running on GCE node
W20260811 17:17:53.012451 8808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:17:53.013181 8798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:17:53.013916 8798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:17:53.015091 8798 hybrid_clock.cc:648] HybridClock initialized: now 1786468673015075 us; error 28 us; skew 500 ppm
Aug 11 17:17:53 dist-test-slave-fd7l krb5kdc[7289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468673, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:17:53.018263 8798 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:17:53.019227 8798 webserver.cc:460] Webserver started at http://127.21.54.130:46137/ using document root <none> and password file <none>
I20260811 17:17:53.019851 8798 fs_manager.cc:359] Metadata directory not provided
I20260811 17:17:53.019953 8798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:17:53.024183 8798 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:17:53.026521 8815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:53.027860 8798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 17:17:53.028031 8798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "29b38e0787754d12a0ee28ed380cfd0d"
format_stamp: "Formatted at 2026-08-11 17:17:10 on dist-test-slave-fd7l"
server_key: "300f8ee1db0729a9b5954a69410675a4"
server_key_iv: "4f07085aee45e845441f1dfb6a66d82a"
server_key_version: "encryptionkey@0"
I20260811 17:17:53.028729 8798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:17:53.064888 8798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:17:53.068790 8798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:17:53.069064 8798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:17:53.069752 8798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:17:53.070685 8798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:17:53.070735 8798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:53.070811 8798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:17:53.070854 8798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:53.081333 8798 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:42201
I20260811 17:17:53.081362 8941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:42201 every 8 connection(s)
I20260811 17:17:53.082448 8798 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:17:53.086794 8944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:53.084920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:58707)
0811 17:17:53.085417 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:53.085531 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":133,"threads_started":1}
W20260811 17:17:53.087283 8942 heartbeater.cc:644] Failed to heartbeat to 127.21.54.190:43111 (0 consecutive failures): Network error: Failed to ping master at 127.21.54.190:43111: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
I20260811 17:17:53.087985 8944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:53.087742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:58025)
0811 17:17:53.087863 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:53.087928 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 17:17:53.088716 8944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:53.088467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:50651)
0811 17:17:53.088615 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:53.088667 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260811 17:17:53.088905 8942 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 17:17:53.089406 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 8798
I20260811 17:17:53.089910 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:40579
--local_ip_for_outbound_sockets=127.21.54.131
--tserver_master_addrs=127.21.54.190:43111
--webserver_port=46243
--webserver_interface=127.21.54.131
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:43423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:17:53.209885 8946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:17:53.210156 8946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:17:53.210201 8946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:17:53.214340 8946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:17:53.214406 8946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:17:53.214466 8946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:17:53.219959 8946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:43423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:40579
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=46243
--tserver_master_addrs=127.21.54.190:43111
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:17:53.221159 8946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:17:53.222193 8946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:17:53.232851 8952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:53.232846 8954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:17:53.232846 8951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:17:53.233244 8946 server_base.cc:1034] running on GCE node
I20260811 17:17:53.233753 8946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:17:53.234437 8946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:17:53.235617 8946 hybrid_clock.cc:648] HybridClock initialized: now 1786468673235598 us; error 30 us; skew 500 ppm
Aug 11 17:17:53 dist-test-slave-fd7l krb5kdc[7289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468673, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:17:53.238549 8946 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:17:53.239497 8946 webserver.cc:460] Webserver started at http://127.21.54.131:46243/ using document root <none> and password file <none>
I20260811 17:17:53.240114 8946 fs_manager.cc:359] Metadata directory not provided
I20260811 17:17:53.240216 8946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:17:53.244422 8946 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:17:53.246665 8961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:53.248080 8946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:17:53.248257 8946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "7a3eef0fd45f40b783d298ce46aee818"
format_stamp: "Formatted at 2026-08-11 17:17:10 on dist-test-slave-fd7l"
server_key: "7005e6ad7aba23ea0f06438f031d5f2e"
server_key_iv: "1df7625b7e605e91be196d54ab2eef42"
server_key_version: "encryptionkey@0"
I20260811 17:17:53.248979 8946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:17:53.263190 8946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:17:53.266753 8946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:17:53.267015 8946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:17:53.267740 8946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:17:53.268743 8946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:17:53.268826 8946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:53.268913 8946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:17:53.268970 8946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:17:53.279726 8946 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:40579
I20260811 17:17:53.279759 9088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:40579 every 8 connection(s)
I20260811 17:17:53.281332 8946 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:17:53.285794 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:53.283838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:35835)
0811 17:17:53.284350 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:53.284470 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":119,"threads_started":1}
W20260811 17:17:53.286270 9089 heartbeater.cc:644] Failed to heartbeat to 127.21.54.190:43111 (0 consecutive failures): Network error: Failed to ping master at 127.21.54.190:43111: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
I20260811 17:17:53.286932 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:53.286699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:49087)
0811 17:17:53.286818 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:53.286877 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 17:17:53.287797 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:53.287580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:35713)
0811 17:17:53.287693 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:53.287749 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 17:17:53.288017 9089 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 17:17:53.288630 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 8946
I20260811 17:17:53.948010 9093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:53.947400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:51243)
0811 17:17:53.947767 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:53.947866 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":80,"threads_started":1}
I20260811 17:17:54.049592 9094 raft_consensus.cc:482] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:17:54.049835 9094 raft_consensus.cc:504] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:54.051118 9094 leader_election.cc:288] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a3eef0fd45f40b783d298ce46aee818 (127.21.54.131:40579), 29b38e0787754d12a0ee28ed380cfd0d (127.21.54.130:42201)
Aug 11 17:17:54 dist-test-slave-fd7l krb5kdc[7289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468672, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.131@KRBTEST.COM
Aug 11 17:17:54 dist-test-slave-fd7l krb5kdc[7289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468672, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.130@KRBTEST.COM
I20260811 17:17:54.072297 9093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:54.051333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:40579 (local address 127.21.54.129:43421)
0811 17:17:54.051562 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:54.051580 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:17:54.051691 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:17:54.053940 (+ 2249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:17:54.053956 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:17:54.054780 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:17:54.055768 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:54.055791 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:54.057756 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:54.057760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:54.059542 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:54.059555 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:54.060668 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:54.060671 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:54.060940 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:54.065196 (+ 4256us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:17:54.065223 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:17:54.066971 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:17:54.069639 (+ 2668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:17:54.069645 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:17:54.069660 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:17:54.070045 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:17:54.070436 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:17:54.070441 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:17:54.070443 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:17:54.070617 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:17:54.071267 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:17:54.071273 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:17:54.071520 (+ 247us) client_negotiation.cc:769] Sending connection context
0811 17:17:54.071712 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 17:17:54.072073 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":407,"spinlock_wait_cycles":38144}
I20260811 17:17:54.072324 9096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:54.051586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:36801 (local address 127.21.54.130:42201)
0811 17:17:54.051978 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:54.051996 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:54.052210 (+ 214us) server_negotiation.cc:407] Connection header received
0811 17:17:54.053000 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:54.053009 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:54.053477 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:54.053810 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:17:54.055977 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:54.057578 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:54.059722 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:54.060484 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:54.060692 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:54.067560 (+ 6868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:17:54.067565 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:17:54.067575 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:17:54.067604 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:17:54.069521 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:54.070198 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:54.070203 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:54.070207 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:54.070336 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:54.070684 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:54.070689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:54.070693 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:54.071131 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:17:54.071273 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:54.071867 (+ 594us) server_negotiation.cc:299] Negotiation successful
0811 17:17:54.072155 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":74,"threads_started":1}
I20260811 17:17:54.072392 9095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:54.051380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:42201 (local address 127.21.54.129:36801)
0811 17:17:54.052095 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:54.052108 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:17:54.052211 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:17:54.053939 (+ 1728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:17:54.053947 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:17:54.054781 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:17:54.055768 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:54.055791 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:54.057760 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:54.057763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:54.059542 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:17:54.059555 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:17:54.060663 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:17:54.060667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:17:54.060940 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:54.065235 (+ 4295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:17:54.065249 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:17:54.067369 (+ 2120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:17:54.069658 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:17:54.069660 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:17:54.069662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:17:54.070058 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:17:54.070473 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:17:54.070476 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:17:54.070478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:17:54.070589 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:17:54.071267 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:17:54.071274 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:17:54.071519 (+ 245us) client_negotiation.cc:769] Sending connection context
0811 17:17:54.071713 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 17:17:54.072161 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":545,"spinlock_wait_cycles":20096,"thread_start_us":73,"threads_started":1}
I20260811 17:17:54.072695 9097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:54.051709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:43421 (local address 127.21.54.131:40579)
0811 17:17:54.051991 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0811 17:17:54.052001 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:17:54.052047 (+ 46us) server_negotiation.cc:407] Connection header received
0811 17:17:54.053000 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:17:54.053009 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:17:54.053478 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:17:54.053810 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:17:54.055977 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:54.057564 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:54.059722 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:17:54.060497 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:17:54.060764 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:17:54.067284 (+ 6520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:17:54.067291 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:17:54.067311 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:17:54.067352 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:17:54.069436 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:54.070175 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:54.070181 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:54.070189 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:54.070317 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:17:54.070739 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:17:54.070741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:17:54.070742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:17:54.071131 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:17:54.071276 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 17:17:54.071845 (+ 569us) server_negotiation.cc:299] Negotiation successful
0811 17:17:54.072161 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":78,"threads_started":1}
W20260811 17:17:54.075160 8665 leader_election.cc:341] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 29b38e0787754d12a0ee28ed380cfd0d (127.21.54.130:42201): Not found: Tablet not found: 472a57249ee94ea289f1e5c151e53b54
W20260811 17:17:54.075416 8666 leader_election.cc:341] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7a3eef0fd45f40b783d298ce46aee818 (127.21.54.131:40579): Not found: Tablet not found: 472a57249ee94ea289f1e5c151e53b54
I20260811 17:17:54.075532 8666 leader_election.cc:302] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51791ee80155440d901587802cb71758; no voters: 29b38e0787754d12a0ee28ed380cfd0d, 7a3eef0fd45f40b783d298ce46aee818
I20260811 17:17:54.075908 9094 raft_consensus.cc:2726] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:17:54.089850 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:54.089450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:47069)
0811 17:17:54.089697 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:54.089751 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":85,"threads_started":1}
I20260811 17:17:54.289182 9099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:54.288723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:41673)
0811 17:17:54.289014 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:54.289078 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":92,"threads_started":1}
I20260811 17:17:54.949415 9100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:54.948979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:42437)
0811 17:17:54.949253 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:54.949313 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":87,"threads_started":1}
I20260811 17:17:55.091198 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:55.090722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:56715)
0811 17:17:55.090995 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:55.091076 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":78,"threads_started":1}
I20260811 17:17:55.290568 9102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:55.290097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:53679)
0811 17:17:55.290405 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:55.290470 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":86,"threads_started":1}
I20260811 17:17:55.950866 9103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:55.950394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:55677)
0811 17:17:55.950701 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:55.950761 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":94,"threads_started":1}
I20260811 17:17:56.071918 9104 raft_consensus.cc:482] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:17:56.072053 9104 raft_consensus.cc:504] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:56.072453 9104 leader_election.cc:288] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a3eef0fd45f40b783d298ce46aee818 (127.21.54.131:40579), 29b38e0787754d12a0ee28ed380cfd0d (127.21.54.130:42201)
W20260811 17:17:56.073381 8665 leader_election.cc:341] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 29b38e0787754d12a0ee28ed380cfd0d (127.21.54.130:42201): Not found: Tablet not found: 472a57249ee94ea289f1e5c151e53b54
W20260811 17:17:56.073488 8666 leader_election.cc:341] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7a3eef0fd45f40b783d298ce46aee818 (127.21.54.131:40579): Not found: Tablet not found: 472a57249ee94ea289f1e5c151e53b54
I20260811 17:17:56.073611 8666 leader_election.cc:302] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51791ee80155440d901587802cb71758; no voters: 29b38e0787754d12a0ee28ed380cfd0d, 7a3eef0fd45f40b783d298ce46aee818
I20260811 17:17:56.073818 9104 raft_consensus.cc:2726] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:17:56.092495 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:56.092060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:42117)
0811 17:17:56.092338 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:56.092400 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":75,"threads_started":1}
I20260811 17:17:56.291872 9106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:56.291449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:33739)
0811 17:17:56.291714 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:56.291779 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":75,"threads_started":1}
I20260811 17:17:56.952381 9107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:56.951887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:52301)
0811 17:17:56.952218 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:56.952280 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":106,"threads_started":1}
I20260811 17:17:57.093819 9108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:57.093365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:46079)
0811 17:17:57.093654 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:57.093715 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":88,"threads_started":1}
I20260811 17:17:57.293269 9109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:57.292800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:41631)
0811 17:17:57.293102 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:57.293168 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":89,"threads_started":1}
I20260811 17:17:57.954093 9110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:57.953475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:33453)
0811 17:17:57.953833 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:57.953890 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":97,"threads_started":1}
I20260811 17:17:58.095301 9111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:58.094780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:34341)
0811 17:17:58.095122 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:58.095195 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":84,"threads_started":1}
I20260811 17:17:58.294680 9112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:58.294214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:40353)
0811 17:17:58.294512 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:58.294576 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":83,"threads_started":1}
I20260811 17:17:58.449344 9113 raft_consensus.cc:482] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:17:58.449659 9113 raft_consensus.cc:504] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:17:58.450279 9113 leader_election.cc:288] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a3eef0fd45f40b783d298ce46aee818 (127.21.54.131:40579), 29b38e0787754d12a0ee28ed380cfd0d (127.21.54.130:42201)
W20260811 17:17:58.450961 8666 leader_election.cc:341] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7a3eef0fd45f40b783d298ce46aee818 (127.21.54.131:40579): Not found: Tablet not found: 472a57249ee94ea289f1e5c151e53b54
W20260811 17:17:58.451078 8665 leader_election.cc:341] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 29b38e0787754d12a0ee28ed380cfd0d (127.21.54.130:42201): Not found: Tablet not found: 472a57249ee94ea289f1e5c151e53b54
I20260811 17:17:58.451148 8665 leader_election.cc:302] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51791ee80155440d901587802cb71758; no voters: 29b38e0787754d12a0ee28ed380cfd0d, 7a3eef0fd45f40b783d298ce46aee818
I20260811 17:17:58.451294 9113 raft_consensus.cc:2726] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:17:58.955410 9114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:58.954948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:35133)
0811 17:17:58.955243 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:58.955307 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":89,"threads_started":1}
I20260811 17:17:59.096658 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:59.096123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:55307)
0811 17:17:59.096396 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:59.096457 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":78,"threads_started":1}
I20260811 17:17:59.296186 9116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:59.295648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:42941)
0811 17:17:59.295975 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:59.296042 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":94,"threads_started":1}
I20260811 17:17:59.957108 9117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:17:59.956455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:52151)
0811 17:17:59.956897 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 17:17:59.956967 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":212,"threads_started":1}
I20260811 17:18:00.098239 9118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:00.097712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:37279)
0811 17:18:00.098057 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:00.098127 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":86,"threads_started":1}
I20260811 17:18:00.297837 9119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:00.297224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:47771)
0811 17:18:00.297666 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:00.297736 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":90,"threads_started":1}
I20260811 17:18:00.958565 9120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:00.958095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:52897)
0811 17:18:00.958401 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:00.958464 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":92,"threads_started":1}
I20260811 17:18:01.099637 9121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:01.099154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:49325)
0811 17:18:01.099450 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:01.099509 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":82,"threads_started":1}
I20260811 17:18:01.299350 9122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:01.298823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:55783)
0811 17:18:01.299173 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:01.299238 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":88,"threads_started":1}
I20260811 17:18:01.645845 9123 raft_consensus.cc:482] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:18:01.646014 9123 raft_consensus.cc:504] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:18:01.646517 9123 leader_election.cc:288] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a3eef0fd45f40b783d298ce46aee818 (127.21.54.131:40579), 29b38e0787754d12a0ee28ed380cfd0d (127.21.54.130:42201)
W20260811 17:18:01.647364 8666 leader_election.cc:341] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7a3eef0fd45f40b783d298ce46aee818 (127.21.54.131:40579): Not found: Tablet not found: 472a57249ee94ea289f1e5c151e53b54
W20260811 17:18:01.647557 8665 leader_election.cc:341] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 29b38e0787754d12a0ee28ed380cfd0d (127.21.54.130:42201): Not found: Tablet not found: 472a57249ee94ea289f1e5c151e53b54
I20260811 17:18:01.647644 8665 leader_election.cc:302] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51791ee80155440d901587802cb71758; no voters: 29b38e0787754d12a0ee28ed380cfd0d, 7a3eef0fd45f40b783d298ce46aee818
I20260811 17:18:01.647805 9123 raft_consensus.cc:2726] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:18:01.960119 9124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:01.959529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:56773)
0811 17:18:01.959884 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:01.959952 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":138,"threads_started":1}
I20260811 17:18:02.101027 9125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:02.100489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:36945)
0811 17:18:02.100848 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:02.100906 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":81,"threads_started":1}
I20260811 17:18:02.300783 9126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:02.300248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:39513)
0811 17:18:02.300574 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:02.300674 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":90,"threads_started":1}
I20260811 17:18:02.961724 9127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:02.961242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:40245)
0811 17:18:02.961563 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:02.961625 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":81,"threads_started":1}
I20260811 17:18:03.102550 9128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:03.102062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:36003)
0811 17:18:03.102377 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:03.102442 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":125,"threads_started":1}
I20260811 17:18:03.302248 9129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:03.301703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:50017)
0811 17:18:03.302041 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:03.302113 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":79,"threads_started":1}
I20260811 17:18:03.963135 9130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:03.962635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:52429)
0811 17:18:03.962914 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:03.962982 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":84,"threads_started":1}
I20260811 17:18:04.103906 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:04.103380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:41353)
0811 17:18:04.103704 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:04.103795 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":82,"threads_started":1}
I20260811 17:18:04.303622 9133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:04.303172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:39749)
0811 17:18:04.303448 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:04.303509 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":80,"threads_started":1}
I20260811 17:18:04.964610 9134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:04.964078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:36827)
0811 17:18:04.964392 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:04.964457 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":113,"threads_started":1}
I20260811 17:18:05.105459 9136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:05.104939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:45861)
0811 17:18:05.105270 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:05.105352 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":98,"threads_started":1}
I20260811 17:18:05.208328 9137 raft_consensus.cc:482] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:18:05.208498 9137 raft_consensus.cc:504] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a3eef0fd45f40b783d298ce46aee818" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40579 } } peers { permanent_uuid: "29b38e0787754d12a0ee28ed380cfd0d" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42201 } } peers { permanent_uuid: "51791ee80155440d901587802cb71758" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 45361 } }
I20260811 17:18:05.209102 9137 leader_election.cc:288] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7a3eef0fd45f40b783d298ce46aee818 (127.21.54.131:40579), 29b38e0787754d12a0ee28ed380cfd0d (127.21.54.130:42201)
W20260811 17:18:05.210088 8666 leader_election.cc:341] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7a3eef0fd45f40b783d298ce46aee818 (127.21.54.131:40579): Not found: Tablet not found: 472a57249ee94ea289f1e5c151e53b54
W20260811 17:18:05.210202 8665 leader_election.cc:341] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 29b38e0787754d12a0ee28ed380cfd0d (127.21.54.130:42201): Not found: Tablet not found: 472a57249ee94ea289f1e5c151e53b54
I20260811 17:18:05.210330 8665 leader_election.cc:302] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51791ee80155440d901587802cb71758; no voters: 29b38e0787754d12a0ee28ed380cfd0d, 7a3eef0fd45f40b783d298ce46aee818
I20260811 17:18:05.210518 9137 raft_consensus.cc:2726] T 472a57249ee94ea289f1e5c151e53b54 P 51791ee80155440d901587802cb71758 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:18:05.305049 9138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:05.304435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:33189)
0811 17:18:05.304836 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:05.304941 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":134,"threads_started":1}
I20260811 17:18:05.966130 9139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:05.965526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:43143)
0811 17:18:05.965944 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:05.966012 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":141,"threads_started":1}
I20260811 17:18:06.107110 9140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:06.106595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:45013)
0811 17:18:06.106933 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:06.106999 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":113,"threads_started":1}
I20260811 17:18:06.306526 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:06.305935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:45099)
0811 17:18:06.306325 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:06.306407 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":98,"threads_started":1}
I20260811 17:18:06.967609 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:06.967054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:46667)
0811 17:18:06.967350 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:06.967485 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":87,"threads_started":1}
I20260811 17:18:07.108692 9143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:07.108108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:34705)
0811 17:18:07.108431 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:07.108581 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":83,"threads_started":1}
I20260811 17:18:07.308030 9144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:07.307503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:53989)
0811 17:18:07.307841 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:07.307919 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":106,"threads_started":1}
I20260811 17:18:07.969550 9146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:07.969032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.129:38601)
0811 17:18:07.969369 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:07.969440 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":89,"threads_started":1}
I20260811 17:18:08.110316 9147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:08.109858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.130:56693)
0811 17:18:08.110149 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:08.110213 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":87,"threads_started":1}
I20260811 17:18:08.294519 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 8602
I20260811 17:18:08.304678 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 8798
I20260811 17:18:08.309722 9148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:08.309204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:43111 (local address 127.21.54.131:42667)
0811 17:18:08.309559 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:08.309622 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.190:43111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":87,"threads_started":1}
I20260811 17:18:08.313786 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 8946
2026-08-11T17:18:08Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (60325 ms)
[----------] 30 tests from SecurityITest (260216 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:08 dist-test-slave-fd7l krb5kdc[9152](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:08 dist-test-slave-fd7l krb5kdc[9152](info): set up 2 sockets
Aug 11 17:18:08 dist-test-slave-fd7l krb5kdc[9152](info): commencing operation
krb5kdc: starting...
W20260811 17:18:09.471443 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:09 dist-test-slave-fd7l krb5kdc[9152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:18:09.616582 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:32967
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:45147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:32967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:09.737357 9168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:09.737752 9168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:09.737845 9168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:09.742043 9168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:09.742161 9168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:09.742201 9168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:09.742261 9168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:09.742300 9168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:09.747975 9168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:45147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:32967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:32967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:09.749344 9168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:09.750499 9168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:09.760329 9177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:09.760331 9175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:09.760377 9168 server_base.cc:1034] running on GCE node
W20260811 17:18:09.760377 9174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:09.761260 9168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:09.762494 9168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:09.763695 9168 hybrid_clock.cc:648] HybridClock initialized: now 1786468689763682 us; error 27 us; skew 500 ppm
Aug 11 17:18:09 dist-test-slave-fd7l krb5kdc[9152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468689, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:09.766842 9168 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:18:09.767896 9168 webserver.cc:460] Webserver started at http://127.21.54.190:37563/ using document root <none> and password file <none>
I20260811 17:18:09.768575 9168 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:09.768677 9168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:09.768927 9168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:09.771129 9168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "cc24cf80cd0f4041a0ede70ac2d0a80d"
format_stamp: "Formatted at 2026-08-11 17:18:09 on dist-test-slave-fd7l"
server_key: "71a7cc594f2c160988db400ee6f82cd6"
server_key_iv: "0f01be70ec77ee12baca59b77364f72c"
server_key_version: "encryptionkey@0"
I20260811 17:18:09.771755 9168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "cc24cf80cd0f4041a0ede70ac2d0a80d"
format_stamp: "Formatted at 2026-08-11 17:18:09 on dist-test-slave-fd7l"
server_key: "71a7cc594f2c160988db400ee6f82cd6"
server_key_iv: "0f01be70ec77ee12baca59b77364f72c"
server_key_version: "encryptionkey@0"
I20260811 17:18:09.775925 9168 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:18:09.778841 9184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:09.780354 9168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:18:09.780786 9168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "cc24cf80cd0f4041a0ede70ac2d0a80d"
format_stamp: "Formatted at 2026-08-11 17:18:09 on dist-test-slave-fd7l"
server_key: "71a7cc594f2c160988db400ee6f82cd6"
server_key_iv: "0f01be70ec77ee12baca59b77364f72c"
server_key_version: "encryptionkey@0"
I20260811 17:18:09.780977 9168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:09.796846 9168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:09.800429 9168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:09.800709 9168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:09.808558 9168 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:32967
I20260811 17:18:09.808584 9242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:32967 every 8 connection(s)
I20260811 17:18:09.809597 9168 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:09.812669 9243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:09.814790 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 9168
I20260811 17:18:09.814957 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:09.815306 21722 external_mini_cluster.cc:1428] Setting key 5b8de67365063c23a2f16a24ccd206fc
I20260811 17:18:09.820067 9243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d: Bootstrap starting.
I20260811 17:18:09.822613 9243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:09.823601 9243 log.cc:824] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d: Log is configured to *not* fsync() on all Append() calls
Aug 11 17:18:09 dist-test-slave-fd7l krb5kdc[9152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:18:09.825994 9243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d: No bootstrap required, opened a new log
I20260811 17:18:09.830752 9243 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc24cf80cd0f4041a0ede70ac2d0a80d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 32967 } }
I20260811 17:18:09.831071 9243 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:09.831168 9243 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc24cf80cd0f4041a0ede70ac2d0a80d, State: Initialized, Role: FOLLOWER
I20260811 17:18:09.831563 9247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:09.817056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34782 (local address 127.21.54.190:32967)
0811 17:18:09.817572 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:09.817589 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:09.817626 (+ 37us) server_negotiation.cc:407] Connection header received
0811 17:18:09.818322 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:09.818341 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:09.818685 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:09.819042 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:09.819918 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:09.821036 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:09.821935 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:09.822436 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:09.822616 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:09.825383 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:09.825389 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:09.825403 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:09.825431 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:09.827539 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:09.828090 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:09.828097 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:09.828102 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:09.828228 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:09.828627 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:09.828631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:09.828635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:09.829250 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:09.829526 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:09.830318 (+ 792us) server_negotiation.cc:299] Negotiation successful
0811 17:18:09.830607 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":169,"threads_started":1}
I20260811 17:18:09.831763 9243 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc24cf80cd0f4041a0ede70ac2d0a80d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 32967 } }
I20260811 17:18:09.831951 9243 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:09.832055 9243 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:09.832170 9243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:09.833299 9243 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc24cf80cd0f4041a0ede70ac2d0a80d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 32967 } }
I20260811 17:18:09.833690 9243 leader_election.cc:302] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc24cf80cd0f4041a0ede70ac2d0a80d; no voters:
I20260811 17:18:09.834108 9243 leader_election.cc:288] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:09.834231 9249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:09.834594 9249 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d [term 1 LEADER]: Becoming Leader. State: Replica: cc24cf80cd0f4041a0ede70ac2d0a80d, State: Running, Role: LEADER
I20260811 17:18:09.834986 9249 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc24cf80cd0f4041a0ede70ac2d0a80d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 32967 } }
I20260811 17:18:09.835568 9243 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:09.837015 9250 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc24cf80cd0f4041a0ede70ac2d0a80d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc24cf80cd0f4041a0ede70ac2d0a80d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 32967 } } }
I20260811 17:18:09.837182 9250 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:09.837566 9251 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc24cf80cd0f4041a0ede70ac2d0a80d. Latest consensus state: current_term: 1 leader_uuid: "cc24cf80cd0f4041a0ede70ac2d0a80d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc24cf80cd0f4041a0ede70ac2d0a80d" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 32967 } } }
I20260811 17:18:09.837649 9258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:09.837669 9251 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:09.840741 9258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:09.846999 9258 catalog_manager.cc:1347] Generated new cluster ID: a4dd4070683b480e9ac943d43e841c43
I20260811 17:18:09.847074 9258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:09.854372 9258 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:09.855346 9258 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:09.859181 9258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d: Generated new TSK 0
I20260811 17:18:09.859947 9258 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
I20260811 17:18:09.896848 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:32967
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:45147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:10.017061 9270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:10.017378 9270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:10.017452 9270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:10.021644 9270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:10.021742 9270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:10.021843 9270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:18:10.027266 9270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:45147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:32967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:10.028419 9270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:10.029520 9270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:10.040104 9276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:10.040141 9275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:10.040268 9270 server_base.cc:1034] running on GCE node
W20260811 17:18:10.040546 9278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:10.040962 9270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:10.041689 9270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:10.042852 9270 hybrid_clock.cc:648] HybridClock initialized: now 1786468690042817 us; error 49 us; skew 500 ppm
Aug 11 17:18:10 dist-test-slave-fd7l krb5kdc[9152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:10.046119 9270 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:18:10.047639 9270 webserver.cc:460] Webserver started at http://127.21.54.129:46493/ using document root <none> and password file <none>
I20260811 17:18:10.048293 9270 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:10.048363 9270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:10.048684 9270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:10.050989 9270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "f1a6816e124744cd9d4da08f2f2442ad"
format_stamp: "Formatted at 2026-08-11 17:18:10 on dist-test-slave-fd7l"
server_key: "63557c39c00e75300d46f3b9eb606278"
server_key_iv: "8128a1d781880a6a0e0ea0e14daa7d61"
server_key_version: "encryptionkey@0"
I20260811 17:18:10.051645 9270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "f1a6816e124744cd9d4da08f2f2442ad"
format_stamp: "Formatted at 2026-08-11 17:18:10 on dist-test-slave-fd7l"
server_key: "63557c39c00e75300d46f3b9eb606278"
server_key_iv: "8128a1d781880a6a0e0ea0e14daa7d61"
server_key_version: "encryptionkey@0"
I20260811 17:18:10.055934 9270 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:18:10.058741 9286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:10.060153 9270 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 17:18:10.060309 9270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "f1a6816e124744cd9d4da08f2f2442ad"
format_stamp: "Formatted at 2026-08-11 17:18:10 on dist-test-slave-fd7l"
server_key: "63557c39c00e75300d46f3b9eb606278"
server_key_iv: "8128a1d781880a6a0e0ea0e14daa7d61"
server_key_version: "encryptionkey@0"
I20260811 17:18:10.060441 9270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:10.100570 9270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:10.104375 9270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:10.104651 9270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:10.105288 9270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:18:10.106168 9270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:18:10.106216 9270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:10.106258 9270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:18:10.106274 9270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:10.115770 9270 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:44681
I20260811 17:18:10.115788 9412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:44681 every 8 connection(s)
I20260811 17:18:10.116866 9270 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:18:10.126484 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 9270
I20260811 17:18:10.126632 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:18:10.127018 21722 external_mini_cluster.cc:1428] Setting key 497f5613ea245f1a276cd993c14a4852
Aug 11 17:18:10 dist-test-slave-fd7l krb5kdc[9152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:18:10.132944 9247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:10.118975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:39921 (local address 127.21.54.190:32967)
0811 17:18:10.119139 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:10.119143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:10.120045 (+ 902us) server_negotiation.cc:407] Connection header received
0811 17:18:10.121547 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:10.121551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:10.121609 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:10.121694 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:10.122998 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.123589 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.124997 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.125266 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.125357 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:10.128553 (+ 3196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:10.128559 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:10.128562 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:10.128591 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:10.130500 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:10.131120 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:10.131129 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:10.131130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:10.131217 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:10.131601 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:10.131604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:10.131606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:10.131805 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:10.131913 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:10.132598 (+ 685us) server_negotiation.cc:299] Negotiation successful
0811 17:18:10.132758 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 17:18:10.133812 9415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:10.119399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:32967 (local address 127.21.54.129:39921)
0811 17:18:10.119872 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:10.119918 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:10.121313 (+ 1395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:10.121870 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:10.121877 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:10.122278 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:10.122835 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:10.122851 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.123771 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.123774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:10.124865 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:10.124877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.125423 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.125425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:10.125574 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:10.126261 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:10.126288 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:10.128317 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:10.130636 (+ 2319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:10.130642 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:10.130659 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:10.130956 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:10.131324 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:10.131328 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:10.131330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:10.131497 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:10.131928 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:10.131936 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:10.132236 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 17:18:10.132451 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 17:18:10.132890 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":119,"threads_started":1}
I20260811 17:18:10.135360 9413 heartbeater.cc:347] Connected to a master server at 127.21.54.190:32967
I20260811 17:18:10.135692 9413 heartbeater.cc:464] Registering TS with master...
I20260811 17:18:10.136345 9413 heartbeater.cc:511] Master 127.21.54.190:32967 requested a full tablet report, sending...
I20260811 17:18:10.138352 9202 ts_manager.cc:194] Registered new tserver with Master: f1a6816e124744cd9d4da08f2f2442ad (127.21.54.129:44681)
WARNING: no policy specified for kudu/127.21.54.130@KRBTEST.COM; defaulting to no policy
I20260811 17:18:10.139716 9202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:39921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
I20260811 17:18:10.161262 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:32967
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:45147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:10.283146 9419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:10.283480 9419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:10.283529 9419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:10.287949 9419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:10.288048 9419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:10.288116 9419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:18:10.294031 9419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:45147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:32967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:10.295291 9419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:10.296461 9419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:10.307315 9427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:10.307353 9429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:10.307308 9426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:10.307717 9419 server_base.cc:1034] running on GCE node
I20260811 17:18:10.308202 9419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:10.309012 9419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:10.310194 9419 hybrid_clock.cc:648] HybridClock initialized: now 1786468690310154 us; error 49 us; skew 500 ppm
Aug 11 17:18:10 dist-test-slave-fd7l krb5kdc[9152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:10.313431 9419 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:18:10.314499 9419 webserver.cc:460] Webserver started at http://127.21.54.130:35849/ using document root <none> and password file <none>
I20260811 17:18:10.315126 9419 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:10.315193 9419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:10.315438 9419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:10.317946 9419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "94e55ab3bc6846bfa70b7a7e916dfcbd"
format_stamp: "Formatted at 2026-08-11 17:18:10 on dist-test-slave-fd7l"
server_key: "c51e98403f16732c26fb35bf8f1390e1"
server_key_iv: "fda33da3647089f354734d6c362d97c8"
server_key_version: "encryptionkey@0"
I20260811 17:18:10.318572 9419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "94e55ab3bc6846bfa70b7a7e916dfcbd"
format_stamp: "Formatted at 2026-08-11 17:18:10 on dist-test-slave-fd7l"
server_key: "c51e98403f16732c26fb35bf8f1390e1"
server_key_iv: "fda33da3647089f354734d6c362d97c8"
server_key_version: "encryptionkey@0"
I20260811 17:18:10.322871 9419 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:18:10.325707 9436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:10.327204 9419 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:10.327378 9419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "94e55ab3bc6846bfa70b7a7e916dfcbd"
format_stamp: "Formatted at 2026-08-11 17:18:10 on dist-test-slave-fd7l"
server_key: "c51e98403f16732c26fb35bf8f1390e1"
server_key_iv: "fda33da3647089f354734d6c362d97c8"
server_key_version: "encryptionkey@0"
I20260811 17:18:10.327528 9419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:10.354314 9419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:10.358315 9419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:10.358596 9419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:10.359328 9419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:18:10.360307 9419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:18:10.360364 9419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:10.360437 9419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:18:10.360483 9419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:10.370177 9419 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:46181
I20260811 17:18:10.370198 9564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:46181 every 8 connection(s)
I20260811 17:18:10.371227 9419 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:18:10.380407 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 9419
I20260811 17:18:10.380626 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:18:10.380951 21722 external_mini_cluster.cc:1428] Setting key ef34b26a153c59060cd11f95a539bacb
Aug 11 17:18:10 dist-test-slave-fd7l krb5kdc[9152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:18:10.386843 9247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:10.373230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:37059 (local address 127.21.54.190:32967)
0811 17:18:10.373381 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:10.373386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:10.374240 (+ 854us) server_negotiation.cc:407] Connection header received
0811 17:18:10.375721 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:10.375724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:10.375784 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:10.375900 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:10.377290 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.377826 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.379025 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.379263 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.379393 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:10.382434 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:10.382438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:10.382441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:10.382474 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:10.384262 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:10.384984 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:10.384991 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:10.384995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:10.385119 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:10.385578 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:10.385581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:10.385583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:10.385775 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:10.385917 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:10.386540 (+ 623us) server_negotiation.cc:299] Negotiation successful
0811 17:18:10.386675 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 17:18:10.387775 9567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:10.373627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:32967 (local address 127.21.54.130:37059)
0811 17:18:10.374084 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:10.374130 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:10.375489 (+ 1359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:10.376081 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:10.376089 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:10.376463 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:10.377068 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:10.377112 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.377966 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.377972 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:10.378874 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:10.378884 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.379450 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.379463 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:10.379634 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:10.380340 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:10.380357 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:10.382249 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:10.384402 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:10.384408 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:10.384424 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:10.384879 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:10.385236 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:10.385241 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:10.385245 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:10.385467 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:10.385919 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:10.385930 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:10.386233 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 17:18:10.386489 (+ 256us) client_negotiation.cc:240] Negotiation successful
0811 17:18:10.386865 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":107,"threads_started":1}
I20260811 17:18:10.389411 9565 heartbeater.cc:347] Connected to a master server at 127.21.54.190:32967
I20260811 17:18:10.389712 9565 heartbeater.cc:464] Registering TS with master...
I20260811 17:18:10.390291 9565 heartbeater.cc:511] Master 127.21.54.190:32967 requested a full tablet report, sending...
I20260811 17:18:10.391492 9202 ts_manager.cc:194] Registered new tserver with Master: 94e55ab3bc6846bfa70b7a7e916dfcbd (127.21.54.130:46181)
I20260811 17:18:10.392143 9202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:37059
WARNING: no policy specified for kudu/127.21.54.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
I20260811 17:18:10.415504 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:32967
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:45147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:10.533445 9571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:10.533744 9571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:10.533789 9571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:10.538870 9571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:10.538968 9571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:10.539073 9571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:18:10.545428 9571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:45147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:32967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:10.546665 9571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:10.547770 9571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:10.558512 9577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:10.558545 9576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:10.558512 9579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:10.558722 9571 server_base.cc:1034] running on GCE node
I20260811 17:18:10.559428 9571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:10.560169 9571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:10.561403 9571 hybrid_clock.cc:648] HybridClock initialized: now 1786468690561348 us; error 73 us; skew 500 ppm
Aug 11 17:18:10 dist-test-slave-fd7l krb5kdc[9152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:10.564954 9571 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:18:10.566079 9571 webserver.cc:460] Webserver started at http://127.21.54.131:45627/ using document root <none> and password file <none>
I20260811 17:18:10.566814 9571 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:10.567008 9571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:10.567320 9571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:10.569645 9571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "b67695354f7342819a0ed59c4176c0ad"
format_stamp: "Formatted at 2026-08-11 17:18:10 on dist-test-slave-fd7l"
server_key: "4e42596e9764531fe5669448244abbf2"
server_key_iv: "7c0569916c3641d0bcf2b1f49aaa217e"
server_key_version: "encryptionkey@0"
I20260811 17:18:10.570250 9571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "b67695354f7342819a0ed59c4176c0ad"
format_stamp: "Formatted at 2026-08-11 17:18:10 on dist-test-slave-fd7l"
server_key: "4e42596e9764531fe5669448244abbf2"
server_key_iv: "7c0569916c3641d0bcf2b1f49aaa217e"
server_key_version: "encryptionkey@0"
I20260811 17:18:10.574635 9571 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:18:10.577493 9586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:10.578897 9571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:10.579066 9571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "b67695354f7342819a0ed59c4176c0ad"
format_stamp: "Formatted at 2026-08-11 17:18:10 on dist-test-slave-fd7l"
server_key: "4e42596e9764531fe5669448244abbf2"
server_key_iv: "7c0569916c3641d0bcf2b1f49aaa217e"
server_key_version: "encryptionkey@0"
I20260811 17:18:10.579205 9571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:10.593457 9571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:10.597158 9571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:10.597388 9571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:10.598048 9571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:18:10.598980 9571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:18:10.599056 9571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:10.599133 9571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:18:10.599176 9571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:10.609767 9571 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:44139
I20260811 17:18:10.609791 9714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:44139 every 8 connection(s)
I20260811 17:18:10.610787 9571 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:18:10.613848 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 9571
I20260811 17:18:10.614012 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:18:10.614346 21722 external_mini_cluster.cc:1428] Setting key 64687344bd4e7935cf4cbe620e6091d8
Aug 11 17:18:10 dist-test-slave-fd7l krb5kdc[9152](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:18:10.626430 9247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:10.612868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:60809 (local address 127.21.54.190:32967)
0811 17:18:10.613014 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:10.613019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:10.613875 (+ 856us) server_negotiation.cc:407] Connection header received
0811 17:18:10.615560 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:10.615563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:10.615624 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:10.615708 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:10.617246 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.617820 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.619059 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.619276 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.619356 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:10.622350 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:10.622353 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:10.622356 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:10.622393 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:10.623963 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:10.624691 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:10.624694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:10.624696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:10.624792 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:10.625205 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:10.625208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:10.625210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:10.625440 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:10.625578 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:10.626161 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 17:18:10.626289 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 17:18:10.627502 9718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:10.613227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:32967 (local address 127.21.54.131:60809)
0811 17:18:10.613712 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:10.613755 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:10.615327 (+ 1572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:10.615919 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:10.615929 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:10.616469 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:10.617100 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:10.617114 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.617967 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.617971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:10.618941 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:10.618951 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.619474 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.619478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:10.619629 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:10.620304 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:10.620322 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:10.622118 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:10.624115 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:10.624121 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:10.624137 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:10.624497 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:10.624907 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:10.624910 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:10.624912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:10.625084 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:10.625599 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:10.625606 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:10.625886 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 17:18:10.626073 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 17:18:10.626386 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":126,"threads_started":1}
I20260811 17:18:10.628804 9716 heartbeater.cc:347] Connected to a master server at 127.21.54.190:32967
I20260811 17:18:10.629082 9716 heartbeater.cc:464] Registering TS with master...
I20260811 17:18:10.629639 9716 heartbeater.cc:511] Master 127.21.54.190:32967 requested a full tablet report, sending...
I20260811 17:18:10.630856 9202 ts_manager.cc:194] Registered new tserver with Master: b67695354f7342819a0ed59c4176c0ad (127.21.54.131:44139)
I20260811 17:18:10.631515 9202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:60809
I20260811 17:18:10.641269 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:18:10.651767 9247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:10.643568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34784 (local address 127.21.54.190:32967)
0811 17:18:10.643737 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:10.643741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:10.643858 (+ 117us) server_negotiation.cc:407] Connection header received
0811 17:18:10.643970 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:10.643972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:10.644020 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:10.644102 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:10.644863 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.645402 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.646336 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.646597 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.646683 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:10.648112 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:10.648115 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:10.648117 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:10.648143 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:10.649813 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:10.650300 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:10.650303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:10.650304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:10.650392 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:10.650748 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:10.650751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:10.650753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:10.651013 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:10.651138 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:10.651406 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 17:18:10.651515 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 17:18:10.655664 9202 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34784:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:18:10.659034 9202 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:18:10.675601 9732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:10.669941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34718 (local address 127.21.54.129:44681)
0811 17:18:10.670347 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:10.670356 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:10.670613 (+ 257us) server_negotiation.cc:407] Connection header received
0811 17:18:10.670701 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:10.670707 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:10.670889 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:10.671054 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:10.671501 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.672611 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.673893 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.674848 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.674983 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:10.675160 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:10.675310 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 17:18:10.675416 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":173,"threads_started":1}
I20260811 17:18:10.675586 9734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:10.669146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:44681 (local address 127.0.0.1:34718)
0811 17:18:10.670402 (+ 1256us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:10.670420 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:10.670511 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:10.671208 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:10.671211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:10.671231 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:10.671331 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:10.671336 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.672746 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.672750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:10.673736 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:10.673748 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.674985 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.674989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:10.675116 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:10.675156 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 17:18:10.675277 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 17:18:10.675416 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1182,"mutex_wait_us":906,"thread_start_us":131,"threads_started":1}
I20260811 17:18:10.676242 9731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:10.669284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40692 (local address 127.21.54.130:46181)
0811 17:18:10.669730 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:10.669745 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:10.669902 (+ 157us) server_negotiation.cc:407] Connection header received
0811 17:18:10.670102 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:10.670111 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:10.670347 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:10.670541 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:10.671072 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.672329 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.674101 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.675022 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.675138 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:10.675276 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:10.675904 (+ 628us) server_negotiation.cc:299] Negotiation successful
0811 17:18:10.676038 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":132,"threads_started":1}
I20260811 17:18:10.675791 9729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:10.669146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:46181 (local address 127.0.0.1:40692)
0811 17:18:10.669754 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:10.669805 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:10.669928 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:10.670753 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:10.670760 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:10.670783 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:10.670926 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:10.670939 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.672495 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.672498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:10.673900 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:10.673913 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.675342 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.675346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:10.675447 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:10.675471 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 17:18:10.675548 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 17:18:10.675624 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":1,"thread_start_us":80,"threads_started":1}
I20260811 17:18:10.676537 9735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:10.670466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:44139 (local address 127.0.0.1:49848)
0811 17:18:10.671246 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:10.671259 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:10.671361 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:10.671908 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:10.671911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:10.671931 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:10.672047 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:10.672054 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.673504 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.673507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:10.674683 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:10.674696 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.676065 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.676069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:10.676178 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:10.676202 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 17:18:10.676282 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 17:18:10.676366 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":3,"thread_start_us":94,"threads_started":1}
I20260811 17:18:10.676945 9736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:10.670643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49848 (local address 127.21.54.131:44139)
0811 17:18:10.671107 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:10.671120 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:10.671490 (+ 370us) server_negotiation.cc:407] Connection header received
0811 17:18:10.671579 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:10.671587 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:10.671758 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:10.671924 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:10.672178 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.673334 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.674838 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.675698 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.675801 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:10.675956 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:10.676671 (+ 715us) server_negotiation.cc:299] Negotiation successful
0811 17:18:10.676774 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":160,"threads_started":1}
I20260811 17:18:10.680008 9341 tablet_service.cc:1467] Processing CreateTablet for tablet 12907ec7d08a42a9a5059492df9b688d (DEFAULT_TABLE table=test-table [id=953e3bad2d5c421ba112fff2fc070e15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:18:10.680017 9490 tablet_service.cc:1467] Processing CreateTablet for tablet 12907ec7d08a42a9a5059492df9b688d (DEFAULT_TABLE table=test-table [id=953e3bad2d5c421ba112fff2fc070e15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:18:10.681289 9490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12907ec7d08a42a9a5059492df9b688d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:10.681290 9341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12907ec7d08a42a9a5059492df9b688d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:10.682132 9640 tablet_service.cc:1467] Processing CreateTablet for tablet 12907ec7d08a42a9a5059492df9b688d (DEFAULT_TABLE table=test-table [id=953e3bad2d5c421ba112fff2fc070e15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:18:10.683316 9640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12907ec7d08a42a9a5059492df9b688d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:10.689347 9738 tablet_bootstrap.cc:492] T 12907ec7d08a42a9a5059492df9b688d P f1a6816e124744cd9d4da08f2f2442ad: Bootstrap starting.
I20260811 17:18:10.689824 9739 tablet_bootstrap.cc:492] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd: Bootstrap starting.
I20260811 17:18:10.692171 9739 tablet_bootstrap.cc:654] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:10.692173 9738 tablet_bootstrap.cc:654] T 12907ec7d08a42a9a5059492df9b688d P f1a6816e124744cd9d4da08f2f2442ad: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:10.692940 9740 tablet_bootstrap.cc:492] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad: Bootstrap starting.
I20260811 17:18:10.693151 9738 log.cc:824] T 12907ec7d08a42a9a5059492df9b688d P f1a6816e124744cd9d4da08f2f2442ad: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:10.693549 9739 log.cc:824] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:10.695336 9740 tablet_bootstrap.cc:654] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:10.695389 9739 tablet_bootstrap.cc:492] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd: No bootstrap required, opened a new log
I20260811 17:18:10.695488 9738 tablet_bootstrap.cc:492] T 12907ec7d08a42a9a5059492df9b688d P f1a6816e124744cd9d4da08f2f2442ad: No bootstrap required, opened a new log
I20260811 17:18:10.695633 9739 ts_tablet_manager.cc:1397] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260811 17:18:10.695691 9738 ts_tablet_manager.cc:1397] T 12907ec7d08a42a9a5059492df9b688d P f1a6816e124744cd9d4da08f2f2442ad: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 17:18:10.696758 9740 log.cc:824] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:10.699076 9740 tablet_bootstrap.cc:492] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad: No bootstrap required, opened a new log
I20260811 17:18:10.699291 9740 ts_tablet_manager.cc:1397] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 17:18:10.699067 9739 raft_consensus.cc:348] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e55ab3bc6846bfa70b7a7e916dfcbd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 46181 } } peers { permanent_uuid: "f1a6816e124744cd9d4da08f2f2442ad" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44681 } } peers { permanent_uuid: "b67695354f7342819a0ed59c4176c0ad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 44139 } }
I20260811 17:18:10.699388 9739 raft_consensus.cc:374] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:10.699446 9739 raft_consensus.cc:729] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94e55ab3bc6846bfa70b7a7e916dfcbd, State: Initialized, Role: FOLLOWER
I20260811 17:18:10.699971 9739 consensus_queue.cc:260] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e55ab3bc6846bfa70b7a7e916dfcbd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 46181 } } peers { permanent_uuid: "f1a6816e124744cd9d4da08f2f2442ad" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44681 } } peers { permanent_uuid: "b67695354f7342819a0ed59c4176c0ad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 44139 } }
I20260811 17:18:10.700156 9738 raft_consensus.cc:348] T 12907ec7d08a42a9a5059492df9b688d P f1a6816e124744cd9d4da08f2f2442ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e55ab3bc6846bfa70b7a7e916dfcbd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 46181 } } peers { permanent_uuid: "f1a6816e124744cd9d4da08f2f2442ad" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44681 } } peers { permanent_uuid: "b67695354f7342819a0ed59c4176c0ad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 44139 } }
I20260811 17:18:10.700420 9738 raft_consensus.cc:374] T 12907ec7d08a42a9a5059492df9b688d P f1a6816e124744cd9d4da08f2f2442ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:10.700469 9738 raft_consensus.cc:729] T 12907ec7d08a42a9a5059492df9b688d P f1a6816e124744cd9d4da08f2f2442ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1a6816e124744cd9d4da08f2f2442ad, State: Initialized, Role: FOLLOWER
I20260811 17:18:10.700963 9738 consensus_queue.cc:260] T 12907ec7d08a42a9a5059492df9b688d P f1a6816e124744cd9d4da08f2f2442ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e55ab3bc6846bfa70b7a7e916dfcbd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 46181 } } peers { permanent_uuid: "f1a6816e124744cd9d4da08f2f2442ad" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44681 } } peers { permanent_uuid: "b67695354f7342819a0ed59c4176c0ad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 44139 } }
I20260811 17:18:10.701146 9739 ts_tablet_manager.cc:1428] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd: Time spent starting tablet: real 0.005s user 0.005s sys 0.002s
I20260811 17:18:10.701543 9565 heartbeater.cc:503] Master 127.21.54.190:32967 was elected leader, sending a full tablet report...
I20260811 17:18:10.701618 9413 heartbeater.cc:503] Master 127.21.54.190:32967 was elected leader, sending a full tablet report...
I20260811 17:18:10.701946 9738 ts_tablet_manager.cc:1428] T 12907ec7d08a42a9a5059492df9b688d P f1a6816e124744cd9d4da08f2f2442ad: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 17:18:10.703254 9740 raft_consensus.cc:348] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e55ab3bc6846bfa70b7a7e916dfcbd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 46181 } } peers { permanent_uuid: "f1a6816e124744cd9d4da08f2f2442ad" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44681 } } peers { permanent_uuid: "b67695354f7342819a0ed59c4176c0ad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 44139 } }
I20260811 17:18:10.703491 9740 raft_consensus.cc:374] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:10.703558 9740 raft_consensus.cc:729] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b67695354f7342819a0ed59c4176c0ad, State: Initialized, Role: FOLLOWER
I20260811 17:18:10.703984 9740 consensus_queue.cc:260] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e55ab3bc6846bfa70b7a7e916dfcbd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 46181 } } peers { permanent_uuid: "f1a6816e124744cd9d4da08f2f2442ad" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44681 } } peers { permanent_uuid: "b67695354f7342819a0ed59c4176c0ad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 44139 } }
I20260811 17:18:10.704854 9716 heartbeater.cc:503] Master 127.21.54.190:32967 was elected leader, sending a full tablet report...
I20260811 17:18:10.704896 9740 ts_tablet_manager.cc:1428] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 17:18:10.839026 9746 raft_consensus.cc:482] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:18:10.839238 9746 raft_consensus.cc:504] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e55ab3bc6846bfa70b7a7e916dfcbd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 46181 } } peers { permanent_uuid: "f1a6816e124744cd9d4da08f2f2442ad" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44681 } } peers { permanent_uuid: "b67695354f7342819a0ed59c4176c0ad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 44139 } }
I20260811 17:18:10.840471 9746 leader_election.cc:288] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94e55ab3bc6846bfa70b7a7e916dfcbd (127.21.54.130:46181), f1a6816e124744cd9d4da08f2f2442ad (127.21.54.129:44681)
I20260811 17:18:10.845014 9718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:10.840835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:44681 (local address 127.21.54.131:58211)
0811 17:18:10.840991 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:10.841009 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:10.841123 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:10.841393 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:10.841395 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:10.841410 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:10.841616 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:10.841625 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.842639 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.842642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:10.843784 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:10.843795 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.844605 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.844608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:10.844701 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:10.844724 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 17:18:10.844796 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 17:18:10.844853 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 17:18:10.845091 9732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:10.841055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:58211 (local address 127.21.54.129:44681)
0811 17:18:10.841178 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:10.841181 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:10.841193 (+ 12us) server_negotiation.cc:407] Connection header received
0811 17:18:10.841245 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:10.841248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:10.841285 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:10.841382 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:10.841763 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.842525 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.843910 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.844397 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.844559 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:10.844614 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:10.844842 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 17:18:10.844935 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 17:18:10.845445 9731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:10.840965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:33547 (local address 127.21.54.130:46181)
0811 17:18:10.841108 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:10.841114 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:10.841887 (+ 773us) server_negotiation.cc:407] Connection header received
0811 17:18:10.841949 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:10.841952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:10.842015 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:10.842112 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:10.842365 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.842931 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.844235 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.844824 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.844938 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:10.844990 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:10.845245 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 17:18:10.845326 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 17:18:10.845873 9747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:10.840835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:46181 (local address 127.21.54.131:33547)
0811 17:18:10.841797 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:10.841808 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:10.841885 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:10.842125 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:10.842128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:10.842158 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:10.842256 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:10.842261 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.843086 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.843089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:10.844124 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:10.844132 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.844947 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.844951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:10.845073 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:10.845106 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 17:18:10.845644 (+ 538us) client_negotiation.cc:240] Negotiation successful
0811 17:18:10.845720 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":893,"mutex_wait_us":572,"thread_start_us":134,"threads_started":1}
I20260811 17:18:10.845818 9363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12907ec7d08a42a9a5059492df9b688d" candidate_uuid: "b67695354f7342819a0ed59c4176c0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1a6816e124744cd9d4da08f2f2442ad" is_pre_election: true
I20260811 17:18:10.846194 9363 raft_consensus.cc:2454] T 12907ec7d08a42a9a5059492df9b688d P f1a6816e124744cd9d4da08f2f2442ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b67695354f7342819a0ed59c4176c0ad in term 0.
I20260811 17:18:10.846562 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12907ec7d08a42a9a5059492df9b688d" candidate_uuid: "b67695354f7342819a0ed59c4176c0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94e55ab3bc6846bfa70b7a7e916dfcbd" is_pre_election: true
I20260811 17:18:10.846760 9588 leader_election.cc:302] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b67695354f7342819a0ed59c4176c0ad, f1a6816e124744cd9d4da08f2f2442ad; no voters:
I20260811 17:18:10.846859 9514 raft_consensus.cc:2454] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b67695354f7342819a0ed59c4176c0ad in term 0.
I20260811 17:18:10.847054 9746 raft_consensus.cc:2781] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:18:10.847187 9746 raft_consensus.cc:482] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:18:10.847246 9746 raft_consensus.cc:3037] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:10.848387 9746 raft_consensus.cc:504] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e55ab3bc6846bfa70b7a7e916dfcbd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 46181 } } peers { permanent_uuid: "f1a6816e124744cd9d4da08f2f2442ad" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44681 } } peers { permanent_uuid: "b67695354f7342819a0ed59c4176c0ad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 44139 } }
I20260811 17:18:10.848827 9746 leader_election.cc:288] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [CANDIDATE]: Term 1 election: Requested vote from peers 94e55ab3bc6846bfa70b7a7e916dfcbd (127.21.54.130:46181), f1a6816e124744cd9d4da08f2f2442ad (127.21.54.129:44681)
I20260811 17:18:10.849208 9514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12907ec7d08a42a9a5059492df9b688d" candidate_uuid: "b67695354f7342819a0ed59c4176c0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94e55ab3bc6846bfa70b7a7e916dfcbd"
I20260811 17:18:10.849300 9363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12907ec7d08a42a9a5059492df9b688d" candidate_uuid: "b67695354f7342819a0ed59c4176c0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1a6816e124744cd9d4da08f2f2442ad"
I20260811 17:18:10.849352 9514 raft_consensus.cc:3037] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:10.849421 9363 raft_consensus.cc:3037] T 12907ec7d08a42a9a5059492df9b688d P f1a6816e124744cd9d4da08f2f2442ad [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:10.850752 9514 raft_consensus.cc:2454] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b67695354f7342819a0ed59c4176c0ad in term 1.
I20260811 17:18:10.850752 9363 raft_consensus.cc:2454] T 12907ec7d08a42a9a5059492df9b688d P f1a6816e124744cd9d4da08f2f2442ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b67695354f7342819a0ed59c4176c0ad in term 1.
I20260811 17:18:10.851301 9592 leader_election.cc:302] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94e55ab3bc6846bfa70b7a7e916dfcbd, b67695354f7342819a0ed59c4176c0ad; no voters:
I20260811 17:18:10.851506 9746 raft_consensus.cc:2781] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:10.851754 9746 raft_consensus.cc:686] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [term 1 LEADER]: Becoming Leader. State: Replica: b67695354f7342819a0ed59c4176c0ad, State: Running, Role: LEADER
I20260811 17:18:10.852137 9746 consensus_queue.cc:237] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e55ab3bc6846bfa70b7a7e916dfcbd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 46181 } } peers { permanent_uuid: "f1a6816e124744cd9d4da08f2f2442ad" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44681 } } peers { permanent_uuid: "b67695354f7342819a0ed59c4176c0ad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 44139 } }
I20260811 17:18:10.855986 9201 catalog_manager.cc:5495] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad reported cstate change: term changed from 0 to 1, leader changed from <none> to b67695354f7342819a0ed59c4176c0ad (127.21.54.131). New cstate: current_term: 1 leader_uuid: "b67695354f7342819a0ed59c4176c0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e55ab3bc6846bfa70b7a7e916dfcbd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 46181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1a6816e124744cd9d4da08f2f2442ad" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b67695354f7342819a0ed59c4176c0ad" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 44139 } health_report { overall_health: HEALTHY } } }
W20260811 17:18:10.861831 9717 tablet.cc:2369] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:18:10.868322 9414 tablet.cc:2369] T 12907ec7d08a42a9a5059492df9b688d P f1a6816e124744cd9d4da08f2f2442ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:18:10.868827 9736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:10.865150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49850 (local address 127.21.54.131:44139)
0811 17:18:10.865297 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:10.865302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:10.865318 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:18:10.865410 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:10.865413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:10.865512 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:10.865637 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:18:10.865894 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.866403 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.867287 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:10.867509 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:10.867620 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:10.867937 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:18:10.868361 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:18:10.868539 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:10.868608 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 17:18:10.868667 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 17:18:10.872478 9566 tablet.cc:2369] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:18:10.875313 9514 raft_consensus.cc:1261] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd [term 1 FOLLOWER]: Refusing update from remote peer b67695354f7342819a0ed59c4176c0ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:18:10.875312 9363 raft_consensus.cc:1261] T 12907ec7d08a42a9a5059492df9b688d P f1a6816e124744cd9d4da08f2f2442ad [term 1 FOLLOWER]: Refusing update from remote peer b67695354f7342819a0ed59c4176c0ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:18:10.876133 9746 consensus_queue.cc:1035] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1a6816e124744cd9d4da08f2f2442ad" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 44681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:18:10.876364 9748 consensus_queue.cc:1035] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "94e55ab3bc6846bfa70b7a7e916dfcbd" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 46181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:18:10.885077 9751 mvcc.cc:205] Tried to move back new op lower bound from 7317375757817208832 to 7317375757733220352. Current Snapshot: MvccSnapshot[applied={T|T < 7317375757817208832}]
I20260811 17:18:10.895105 9200 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:18:10.895277 9200 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:18:10.898201 9200 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc24cf80cd0f4041a0ede70ac2d0a80d: Sending DeleteTablet for 3 replicas of tablet 12907ec7d08a42a9a5059492df9b688d
I20260811 17:18:10.899217 9490 tablet_service.cc:1514] Processing DeleteTablet for tablet 12907ec7d08a42a9a5059492df9b688d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:18:10 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:40692
I20260811 17:18:10.899392 9341 tablet_service.cc:1514] Processing DeleteTablet for tablet 12907ec7d08a42a9a5059492df9b688d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:18:10 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:34718
I20260811 17:18:10.899996 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 9270
I20260811 17:18:10.899958 9640 tablet_service.cc:1514] Processing DeleteTablet for tablet 12907ec7d08a42a9a5059492df9b688d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:18:10 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:49848
I20260811 17:18:10.900406 9763 tablet_replica.cc:330] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd: stopping tablet replica
I20260811 17:18:10.900810 9763 raft_consensus.cc:2229] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:18:10.901187 9763 raft_consensus.cc:2258] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:18:10.901870 9765 tablet_replica.cc:330] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad: stopping tablet replica
I20260811 17:18:10.902201 9765 raft_consensus.cc:2229] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:18:10.903057 9765 raft_consensus.cc:2258] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:18:10.905218 9765 ts_tablet_manager.cc:1905] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:18:10.906028 9763 ts_tablet_manager.cc:1905] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:18:10.908980 9763 ts_tablet_manager.cc:1918] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:18:10.908982 9765 ts_tablet_manager.cc:1918] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:18:10.909093 9765 log.cc:1196] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/12907ec7d08a42a9a5059492df9b688d
I20260811 17:18:10.909247 9763 log.cc:1196] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/12907ec7d08a42a9a5059492df9b688d
I20260811 17:18:10.909468 9765 ts_tablet_manager.cc:1939] T 12907ec7d08a42a9a5059492df9b688d P b67695354f7342819a0ed59c4176c0ad: Deleting consensus metadata
I20260811 17:18:10.909771 9763 ts_tablet_manager.cc:1939] T 12907ec7d08a42a9a5059492df9b688d P 94e55ab3bc6846bfa70b7a7e916dfcbd: Deleting consensus metadata
I20260811 17:18:10.910893 9186 catalog_manager.cc:4841] TS b67695354f7342819a0ed59c4176c0ad (127.21.54.131:44139): tablet 12907ec7d08a42a9a5059492df9b688d (table test-table [id=953e3bad2d5c421ba112fff2fc070e15]) successfully deleted
I20260811 17:18:10.911069 9188 catalog_manager.cc:4841] TS 94e55ab3bc6846bfa70b7a7e916dfcbd (127.21.54.130:46181): tablet 12907ec7d08a42a9a5059492df9b688d (table test-table [id=953e3bad2d5c421ba112fff2fc070e15]) successfully deleted
W20260811 17:18:10.912611 9186 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.54.129:44681 (error 108)
I20260811 17:18:10.913622 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 9419
I20260811 17:18:10.913849 9734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:10.913478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:44681 (local address 127.0.0.1:34722)
0811 17:18:10.913648 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:10.913738 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.129:44681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 17:18:10.916275 9186 catalog_manager.cc:4584] TS f1a6816e124744cd9d4da08f2f2442ad (127.21.54.129:44681): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 12907ec7d08a42a9a5059492df9b688d: Network error: Client connection negotiation failed: client connection to 127.21.54.129:44681: connect: Connection refused (error 111)
I20260811 17:18:10.927357 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 9571
I20260811 17:18:10.937197 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 9168
2026-08-11T17:18:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2621 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T17:18:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:10Z Disabled control of system clock
I20260811 17:18:10.973840 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:37053
--webserver_interface=127.21.54.190
--webserver_port=0
--builtin_ntp_servers=127.21.54.148:39239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:37053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 17:18:11.089429 9770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:11.089725 9770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:11.089762 9770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:11.093783 9770 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:11.093852 9770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:11.093868 9770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:11.093885 9770 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:11.093900 9770 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:11.098801 9770 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:37053
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:37053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:11.099936 9770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:11.101387 9770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:11.110467 9778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:11.110534 9770 server_base.cc:1034] running on GCE node
W20260811 17:18:11.110463 9775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:11.110424 9776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:11.111526 9770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:11.112846 9770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:11.114068 9770 hybrid_clock.cc:648] HybridClock initialized: now 1786468691114056 us; error 36 us; skew 500 ppm
I20260811 17:18:11.116199 9770 webserver.cc:460] Webserver started at http://127.21.54.190:39897/ using document root <none> and password file <none>
I20260811 17:18:11.116884 9770 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:11.116987 9770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:11.117240 9770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:11.119341 9770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "3c353cebb7dc45aeafc25ea15516f045"
format_stamp: "Formatted at 2026-08-11 17:18:11 on dist-test-slave-fd7l"
server_key: "c70f841799e795bc03a77bfcdf5dd1eb"
server_key_iv: "92bff9a3d8712417a860b30e8ebdb0f3"
server_key_version: "encryptionkey@0"
I20260811 17:18:11.119936 9770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "3c353cebb7dc45aeafc25ea15516f045"
format_stamp: "Formatted at 2026-08-11 17:18:11 on dist-test-slave-fd7l"
server_key: "c70f841799e795bc03a77bfcdf5dd1eb"
server_key_iv: "92bff9a3d8712417a860b30e8ebdb0f3"
server_key_version: "encryptionkey@0"
I20260811 17:18:11.123845 9770 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:18:11.126470 9784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:11.127832 9770 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 17:18:11.128011 9770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "3c353cebb7dc45aeafc25ea15516f045"
format_stamp: "Formatted at 2026-08-11 17:18:11 on dist-test-slave-fd7l"
server_key: "c70f841799e795bc03a77bfcdf5dd1eb"
server_key_iv: "92bff9a3d8712417a860b30e8ebdb0f3"
server_key_version: "encryptionkey@0"
I20260811 17:18:11.128151 9770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:11.143850 9770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:11.144680 9770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:11.144923 9770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:11.152725 9770 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:37053
I20260811 17:18:11.152742 9843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:37053 every 8 connection(s)
I20260811 17:18:11.153779 9770 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:11.157169 9844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:11.161099 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 9770
I20260811 17:18:11.161257 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:11.161594 21722 external_mini_cluster.cc:1428] Setting key ed25ae3db3cdbf96298d51d6f577fbc1
I20260811 17:18:11.163815 9844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045: Bootstrap starting.
I20260811 17:18:11.166208 9844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:11.166970 9844 log.cc:824] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:11.169059 9844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045: No bootstrap required, opened a new log
I20260811 17:18:11.171866 9847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:11.163304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37234 (local address 127.21.54.190:37053)
0811 17:18:11.164090 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:11.164111 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:11.164150 (+ 39us) server_negotiation.cc:407] Connection header received
0811 17:18:11.164884 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:11.164905 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:11.165307 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:11.165714 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:11.166161 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.167282 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.168235 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.168807 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.168989 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:11.169903 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:11.169908 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:11.169921 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:11.170325 (+ 404us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:11.170382 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:11.170404 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:11.170535 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:11.170645 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 17:18:11.170920 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":165,"threads_started":1}
I20260811 17:18:11.172577 9844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c353cebb7dc45aeafc25ea15516f045" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37053 } }
I20260811 17:18:11.172842 9844 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:11.172919 9844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c353cebb7dc45aeafc25ea15516f045, State: Initialized, Role: FOLLOWER
I20260811 17:18:11.173413 9844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c353cebb7dc45aeafc25ea15516f045" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37053 } }
I20260811 17:18:11.173573 9844 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:11.173633 9844 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:11.173727 9844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:11.174819 9844 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c353cebb7dc45aeafc25ea15516f045" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37053 } }
I20260811 17:18:11.175117 9844 leader_election.cc:302] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c353cebb7dc45aeafc25ea15516f045; no voters:
I20260811 17:18:11.175384 9844 leader_election.cc:288] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:11.175529 9849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:11.175806 9849 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045 [term 1 LEADER]: Becoming Leader. State: Replica: 3c353cebb7dc45aeafc25ea15516f045, State: Running, Role: LEADER
I20260811 17:18:11.176143 9849 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c353cebb7dc45aeafc25ea15516f045" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37053 } }
I20260811 17:18:11.176674 9844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:11.177983 9851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c353cebb7dc45aeafc25ea15516f045. Latest consensus state: current_term: 1 leader_uuid: "3c353cebb7dc45aeafc25ea15516f045" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c353cebb7dc45aeafc25ea15516f045" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37053 } } }
I20260811 17:18:11.178117 9851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:11.177964 9850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c353cebb7dc45aeafc25ea15516f045" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c353cebb7dc45aeafc25ea15516f045" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37053 } } }
I20260811 17:18:11.178275 9850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045 [sys.catalog]: This master's current role is: LEADER
W20260811 17:18:11.181734 9866 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:18:11.181802 9866 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:18:11.181886 9860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:11.182664 9860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:11.187925 9860 catalog_manager.cc:1347] Generated new cluster ID: a9b03b9bbf4c4732902e29c86999edda
I20260811 17:18:11.187994 9860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:11.203084 9860 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:11.204439 9860 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:11.220180 9860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045: Generated new TSK 0
I20260811 17:18:11.221138 9860 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:11.227727 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:37053
--builtin_ntp_servers=127.21.54.148:39239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 17:18:11.348320 9871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:11.348656 9871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:11.348698 9871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:11.352661 9871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:11.352718 9871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:11.352773 9871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:18:11.357762 9871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:37053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:11.358886 9871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:11.360239 9871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:11.370735 9880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:11.370828 9871 server_base.cc:1034] running on GCE node
W20260811 17:18:11.370718 9877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:11.370718 9876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:11.371502 9871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:11.372179 9871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:11.373368 9871 hybrid_clock.cc:648] HybridClock initialized: now 1786468691373321 us; error 56 us; skew 500 ppm
I20260811 17:18:11.375612 9871 webserver.cc:460] Webserver started at http://127.21.54.129:39963/ using document root <none> and password file <none>
I20260811 17:18:11.376315 9871 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:11.376390 9871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:11.376700 9871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:11.379313 9871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "6d39015bc5ef48019f6a3ccff6a9deb0"
format_stamp: "Formatted at 2026-08-11 17:18:11 on dist-test-slave-fd7l"
server_key: "36f3edc1aac13a43333c6707f8f069f7"
server_key_iv: "1b73833d817f34f480a2b6bc8041f44e"
server_key_version: "encryptionkey@0"
I20260811 17:18:11.379998 9871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "6d39015bc5ef48019f6a3ccff6a9deb0"
format_stamp: "Formatted at 2026-08-11 17:18:11 on dist-test-slave-fd7l"
server_key: "36f3edc1aac13a43333c6707f8f069f7"
server_key_iv: "1b73833d817f34f480a2b6bc8041f44e"
server_key_version: "encryptionkey@0"
I20260811 17:18:11.384223 9871 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 17:18:11.387274 9887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:11.388782 9871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:11.388937 9871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "6d39015bc5ef48019f6a3ccff6a9deb0"
format_stamp: "Formatted at 2026-08-11 17:18:11 on dist-test-slave-fd7l"
server_key: "36f3edc1aac13a43333c6707f8f069f7"
server_key_iv: "1b73833d817f34f480a2b6bc8041f44e"
server_key_version: "encryptionkey@0"
I20260811 17:18:11.389070 9871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:11.401440 9871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:11.402292 9871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:11.402551 9871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:11.403264 9871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:18:11.404410 9871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:18:11.404493 9871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:11.404619 9871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:18:11.404668 9871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:11.415354 9871 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:42677
I20260811 17:18:11.415376 10010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:42677 every 8 connection(s)
I20260811 17:18:11.416487 9871 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:18:11.426496 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 9871
I20260811 17:18:11.426666 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:18:11.426810 9847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:11.418546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:53821 (local address 127.21.54.190:37053)
0811 17:18:11.418717 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:11.418721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:11.419808 (+ 1087us) server_negotiation.cc:407] Connection header received
0811 17:18:11.421462 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:11.421468 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:11.421545 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:11.421656 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:11.422638 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.423208 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.424432 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.424705 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.424836 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:11.426104 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:11.426108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:11.426110 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:11.426124 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:11.426170 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:11.426184 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:11.426274 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:11.426486 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 17:18:11.426618 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 17:18:11.427100 21722 external_mini_cluster.cc:1428] Setting key 1cd9c7eb80eb106919164d2dd2da43dd
I20260811 17:18:11.427881 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:11.418936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:37053 (local address 127.21.54.129:53821)
0811 17:18:11.419639 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:11.419682 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:11.421159 (+ 1477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:11.421850 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:11.421857 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:11.421953 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:11.422477 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:11.422491 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.423368 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.423372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:11.424282 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:11.424293 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.424883 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.424888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:11.425087 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:11.425766 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:18:11.425768 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:11.425814 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:18:11.425884 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:11.426314 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:11.426324 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:11.426330 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 17:18:11.426457 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 17:18:11.426818 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":118,"threads_started":1}
I20260811 17:18:11.429329 10011 heartbeater.cc:347] Connected to a master server at 127.21.54.190:37053
I20260811 17:18:11.429622 10011 heartbeater.cc:464] Registering TS with master...
I20260811 17:18:11.429625 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:37053
--builtin_ntp_servers=127.21.54.148:39239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 17:18:11.430219 10011 heartbeater.cc:511] Master 127.21.54.190:37053 requested a full tablet report, sending...
I20260811 17:18:11.431872 9802 ts_manager.cc:194] Registered new tserver with Master: 6d39015bc5ef48019f6a3ccff6a9deb0 (127.21.54.129:42677)
I20260811 17:18:11.433404 9802 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.54.129:53821
W20260811 17:18:11.549546 10014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:11.549868 10014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:11.549943 10014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:11.554083 10014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:11.554188 10014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:11.554283 10014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:18:11.559125 10014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:37053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:11.560283 10014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:11.561744 10014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:11.571636 10024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:11.571645 10019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:11.571843 10014 server_base.cc:1034] running on GCE node
W20260811 17:18:11.571655 10020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:11.572314 10014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:11.573158 10014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:11.574349 10014 hybrid_clock.cc:648] HybridClock initialized: now 1786468691574305 us; error 57 us; skew 500 ppm
I20260811 17:18:11.576620 10014 webserver.cc:460] Webserver started at http://127.21.54.130:38797/ using document root <none> and password file <none>
I20260811 17:18:11.577260 10014 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:11.577327 10014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:11.577606 10014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:11.579909 10014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "ae43123eb6ab4457925b7c99451b323f"
format_stamp: "Formatted at 2026-08-11 17:18:11 on dist-test-slave-fd7l"
server_key: "5a9317fbb4b5c26ebb8463bced9b7d0d"
server_key_iv: "cdb8fb8fb9e20de4f88209aa22fe91d2"
server_key_version: "encryptionkey@0"
I20260811 17:18:11.580629 10014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "ae43123eb6ab4457925b7c99451b323f"
format_stamp: "Formatted at 2026-08-11 17:18:11 on dist-test-slave-fd7l"
server_key: "5a9317fbb4b5c26ebb8463bced9b7d0d"
server_key_iv: "cdb8fb8fb9e20de4f88209aa22fe91d2"
server_key_version: "encryptionkey@0"
I20260811 17:18:11.584580 10014 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:18:11.587324 10034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:11.588754 10014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:11.588896 10014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "ae43123eb6ab4457925b7c99451b323f"
format_stamp: "Formatted at 2026-08-11 17:18:11 on dist-test-slave-fd7l"
server_key: "5a9317fbb4b5c26ebb8463bced9b7d0d"
server_key_iv: "cdb8fb8fb9e20de4f88209aa22fe91d2"
server_key_version: "encryptionkey@0"
I20260811 17:18:11.589056 10014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:11.605712 10014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:11.606494 10014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:11.606711 10014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:11.607367 10014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:18:11.608376 10014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:18:11.608460 10014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:11.608572 10014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:18:11.608615 10014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:11.617995 10014 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:42023
I20260811 17:18:11.618059 10156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:42023 every 8 connection(s)
I20260811 17:18:11.619056 10014 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:18:11.628252 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 10014
I20260811 17:18:11.628435 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:18:11.628757 21722 external_mini_cluster.cc:1428] Setting key 70b93dd19e9fe84491ae4996c7b15727
I20260811 17:18:11.628908 9847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:11.621136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:34687 (local address 127.21.54.190:37053)
0811 17:18:11.621302 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:11.621307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:11.622480 (+ 1173us) server_negotiation.cc:407] Connection header received
0811 17:18:11.623894 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:11.623896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:11.623947 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:11.624029 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:11.625082 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.625655 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.626768 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.626983 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.627077 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:11.628284 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:11.628287 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:11.628289 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:11.628302 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:11.628321 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:11.628328 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:11.628428 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:11.628694 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 17:18:11.628759 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 17:18:11.629925 10159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:11.621585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:37053 (local address 127.21.54.130:34687)
0811 17:18:11.622321 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:11.622365 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:11.623693 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:11.624199 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:11.624205 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:11.624286 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:11.624943 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:11.624957 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.625769 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.625771 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:11.626650 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:11.626659 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.627149 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.627152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:11.627305 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:11.627957 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:18:11.627961 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:11.628020 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:18:11.628110 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:11.628430 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:11.628444 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:11.628454 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 17:18:11.628678 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 17:18:11.628972 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":132,"threads_started":1}
I20260811 17:18:11.631093 10157 heartbeater.cc:347] Connected to a master server at 127.21.54.190:37053
I20260811 17:18:11.631088 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:37053
--builtin_ntp_servers=127.21.54.148:39239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 17:18:11.631342 10157 heartbeater.cc:464] Registering TS with master...
I20260811 17:18:11.631992 10157 heartbeater.cc:511] Master 127.21.54.190:37053 requested a full tablet report, sending...
I20260811 17:18:11.633222 9802 ts_manager.cc:194] Registered new tserver with Master: ae43123eb6ab4457925b7c99451b323f (127.21.54.130:42023)
I20260811 17:18:11.633812 9802 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.54.130:34687
W20260811 17:18:11.751696 10160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:11.751994 10160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:11.752034 10160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:11.756213 10160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:11.756296 10160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:11.756352 10160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:18:11.761502 10160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:37053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:11.762822 10160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:11.764312 10160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:11.774600 10167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:11.774684 10169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:11.774693 10166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:11.775019 10160 server_base.cc:1034] running on GCE node
I20260811 17:18:11.775476 10160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:11.776218 10160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:11.777395 10160 hybrid_clock.cc:648] HybridClock initialized: now 1786468691777353 us; error 55 us; skew 500 ppm
I20260811 17:18:11.779596 10160 webserver.cc:460] Webserver started at http://127.21.54.131:39311/ using document root <none> and password file <none>
I20260811 17:18:11.780172 10160 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:11.780233 10160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:11.780500 10160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:11.782934 10160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "dc392b007b9445a5b1db42289c5b56ee"
format_stamp: "Formatted at 2026-08-11 17:18:11 on dist-test-slave-fd7l"
server_key: "c06e5d73ff34059054b5d5b27133decd"
server_key_iv: "a72a53c90dfeccdf7aca1b4727a04caf"
server_key_version: "encryptionkey@0"
I20260811 17:18:11.783499 10160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "dc392b007b9445a5b1db42289c5b56ee"
format_stamp: "Formatted at 2026-08-11 17:18:11 on dist-test-slave-fd7l"
server_key: "c06e5d73ff34059054b5d5b27133decd"
server_key_iv: "a72a53c90dfeccdf7aca1b4727a04caf"
server_key_version: "encryptionkey@0"
I20260811 17:18:11.787385 10160 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:18:11.790228 10175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:11.791760 10160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:18:11.791925 10160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "dc392b007b9445a5b1db42289c5b56ee"
format_stamp: "Formatted at 2026-08-11 17:18:11 on dist-test-slave-fd7l"
server_key: "c06e5d73ff34059054b5d5b27133decd"
server_key_iv: "a72a53c90dfeccdf7aca1b4727a04caf"
server_key_version: "encryptionkey@0"
I20260811 17:18:11.792062 10160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:11.812496 10160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:11.813360 10160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:11.813726 10160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:11.814451 10160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:18:11.815550 10160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:18:11.815630 10160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:11.815711 10160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:18:11.815766 10160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:11.825912 10160 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:35379
I20260811 17:18:11.825932 10303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:35379 every 8 connection(s)
I20260811 17:18:11.826941 10160 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:18:11.829619 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 10160
I20260811 17:18:11.829761 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:18:11.830108 21722 external_mini_cluster.cc:1428] Setting key ea447759d51e2fba7e9fff985b19f4e7
I20260811 17:18:11.837481 9847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:11.829005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:46501 (local address 127.21.54.190:37053)
0811 17:18:11.829171 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:11.829176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:11.830401 (+ 1225us) server_negotiation.cc:407] Connection header received
0811 17:18:11.832155 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:11.832158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:11.832237 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:11.832331 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:11.833459 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.833993 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.835217 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.835445 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.835526 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:11.837008 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:11.837011 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:11.837013 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:11.837026 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:11.837046 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:11.837053 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:11.837131 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:11.837303 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 17:18:11.837359 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 17:18:11.838578 10307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:11.829409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:37053 (local address 127.21.54.131:46501)
0811 17:18:11.830236 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:11.830289 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:11.831903 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:11.832682 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:11.832689 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:11.832774 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:11.833310 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:11.833325 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.834122 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.834125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:11.835089 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:11.835100 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.835749 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.835752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:11.835917 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:11.836677 (+ 760us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:18:11.836679 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:11.836725 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:18:11.836801 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:11.837150 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:11.837158 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:11.837163 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 17:18:11.837283 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 17:18:11.837651 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":149,"threads_started":1}
I20260811 17:18:11.839905 10304 heartbeater.cc:347] Connected to a master server at 127.21.54.190:37053
I20260811 17:18:11.840195 10304 heartbeater.cc:464] Registering TS with master...
I20260811 17:18:11.840843 10304 heartbeater.cc:511] Master 127.21.54.190:37053 requested a full tablet report, sending...
I20260811 17:18:11.842038 9802 ts_manager.cc:194] Registered new tserver with Master: dc392b007b9445a5b1db42289c5b56ee (127.21.54.131:35379)
I20260811 17:18:11.842680 9802 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.54.131:46501
I20260811 17:18:11.844581 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:18:11.850485 9847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:11.846757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37240 (local address 127.21.54.190:37053)
0811 17:18:11.846914 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:11.846919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:11.847118 (+ 199us) server_negotiation.cc:407] Connection header received
0811 17:18:11.847224 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:11.847226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:11.847267 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:11.847334 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:11.847597 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.848108 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.849034 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.849255 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.849347 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:11.850030 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:11.850033 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:11.850035 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:11.850047 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:11.850067 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:11.850074 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:11.850132 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:11.850231 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 17:18:11.850280 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 17:18:11.853365 9802 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:18:11.856419 9802 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:18:11.871929 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:11.865819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:42677 (local address 127.0.0.1:34984)
0811 17:18:11.866201 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:11.866259 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:11.866395 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:11.867290 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:11.867295 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:11.867321 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:11.867475 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:11.867485 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.869244 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.869247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:11.870553 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:11.870563 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.871363 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.871367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:11.871473 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:11.871522 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 17:18:11.871643 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 17:18:11.871768 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":138,"thread_start_us":117,"threads_started":1}
I20260811 17:18:11.871933 10319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:11.866052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34984 (local address 127.21.54.129:42677)
0811 17:18:11.866757 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:11.866769 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:11.866797 (+ 28us) server_negotiation.cc:407] Connection header received
0811 17:18:11.866874 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:11.866882 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:11.867103 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:11.867733 (+ 630us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:11.867873 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.869118 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.870670 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.871220 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.871361 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:11.871569 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:11.871672 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 17:18:11.871768 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":121,"threads_started":1}
I20260811 17:18:11.872267 10320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:11.866155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:42023 (local address 127.0.0.1:41090)
0811 17:18:11.866854 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:11.866867 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:11.866960 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:11.867575 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:11.867578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:11.867598 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:11.867711 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:11.867717 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.869008 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.869014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:11.870460 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:11.870472 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.871772 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.871776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:11.871916 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:11.871957 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 17:18:11.872040 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 17:18:11.872133 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":37,"thread_start_us":117,"threads_started":1}
I20260811 17:18:11.872610 10321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:11.866587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:35379 (local address 127.0.0.1:54854)
0811 17:18:11.867162 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:11.867174 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:11.867243 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:11.868073 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:11.868076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:11.868094 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:11.868169 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:11.868175 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.869477 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.869481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:11.870583 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:11.870593 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.871772 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.871776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:11.871920 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:11.871963 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 17:18:11.872381 (+ 418us) client_negotiation.cc:240] Negotiation successful
0811 17:18:11.872459 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":67,"thread_start_us":97,"threads_started":1}
I20260811 17:18:11.872750 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:11.866670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41090 (local address 127.21.54.130:42023)
0811 17:18:11.867098 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:11.867110 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:11.867132 (+ 22us) server_negotiation.cc:407] Connection header received
0811 17:18:11.867213 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:11.867218 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:11.867400 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:11.867846 (+ 446us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:11.867947 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.868881 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.870595 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.871413 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.871526 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:11.871704 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:11.872452 (+ 748us) server_negotiation.cc:299] Negotiation successful
0811 17:18:11.872579 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":121,"threads_started":1}
I20260811 17:18:11.873078 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:11.866991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54854 (local address 127.21.54.131:35379)
0811 17:18:11.867640 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:11.867657 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:11.867680 (+ 23us) server_negotiation.cc:407] Connection header received
0811 17:18:11.867751 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:11.867756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:11.867925 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:11.868083 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:11.868291 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.869368 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.870719 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.871438 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.871537 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:11.871703 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:11.872779 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0811 17:18:11.872886 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":115,"threads_started":1}
I20260811 17:18:11.875409 10084 tablet_service.cc:1467] Processing CreateTablet for tablet 5e11e667e04040faba80d1d798cc0435 (DEFAULT_TABLE table=test-table [id=0a17553ec7cc4e0bbfde4d271e35eb5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:18:11.875418 9937 tablet_service.cc:1467] Processing CreateTablet for tablet 5e11e667e04040faba80d1d798cc0435 (DEFAULT_TABLE table=test-table [id=0a17553ec7cc4e0bbfde4d271e35eb5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:18:11.876798 10084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e11e667e04040faba80d1d798cc0435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:11.876828 9937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e11e667e04040faba80d1d798cc0435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:11.876737 10233 tablet_service.cc:1467] Processing CreateTablet for tablet 5e11e667e04040faba80d1d798cc0435 (DEFAULT_TABLE table=test-table [id=0a17553ec7cc4e0bbfde4d271e35eb5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:18:11.877890 10233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e11e667e04040faba80d1d798cc0435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:11.883494 10325 tablet_bootstrap.cc:492] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f: Bootstrap starting.
I20260811 17:18:11.884893 10326 tablet_bootstrap.cc:492] T 5e11e667e04040faba80d1d798cc0435 P 6d39015bc5ef48019f6a3ccff6a9deb0: Bootstrap starting.
I20260811 17:18:11.886276 10325 tablet_bootstrap.cc:654] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:11.886963 10327 tablet_bootstrap.cc:492] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee: Bootstrap starting.
I20260811 17:18:11.887212 10326 tablet_bootstrap.cc:654] T 5e11e667e04040faba80d1d798cc0435 P 6d39015bc5ef48019f6a3ccff6a9deb0: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:11.887269 10325 log.cc:824] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:11.888000 10326 log.cc:824] T 5e11e667e04040faba80d1d798cc0435 P 6d39015bc5ef48019f6a3ccff6a9deb0: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:11.889267 10327 tablet_bootstrap.cc:654] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:11.889652 10325 tablet_bootstrap.cc:492] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f: No bootstrap required, opened a new log
I20260811 17:18:11.889878 10325 ts_tablet_manager.cc:1397] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 17:18:11.890041 10326 tablet_bootstrap.cc:492] T 5e11e667e04040faba80d1d798cc0435 P 6d39015bc5ef48019f6a3ccff6a9deb0: No bootstrap required, opened a new log
I20260811 17:18:11.890138 10327 log.cc:824] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:11.890256 10326 ts_tablet_manager.cc:1397] T 5e11e667e04040faba80d1d798cc0435 P 6d39015bc5ef48019f6a3ccff6a9deb0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 17:18:11.892354 10327 tablet_bootstrap.cc:492] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee: No bootstrap required, opened a new log
I20260811 17:18:11.892666 10327 ts_tablet_manager.cc:1397] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 17:18:11.894548 10326 raft_consensus.cc:348] T 5e11e667e04040faba80d1d798cc0435 P 6d39015bc5ef48019f6a3ccff6a9deb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d39015bc5ef48019f6a3ccff6a9deb0" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42677 } } peers { permanent_uuid: "dc392b007b9445a5b1db42289c5b56ee" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35379 } } peers { permanent_uuid: "ae43123eb6ab4457925b7c99451b323f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42023 } }
I20260811 17:18:11.894549 10325 raft_consensus.cc:348] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d39015bc5ef48019f6a3ccff6a9deb0" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42677 } } peers { permanent_uuid: "dc392b007b9445a5b1db42289c5b56ee" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35379 } } peers { permanent_uuid: "ae43123eb6ab4457925b7c99451b323f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42023 } }
I20260811 17:18:11.894850 10325 raft_consensus.cc:374] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:11.894850 10326 raft_consensus.cc:374] T 5e11e667e04040faba80d1d798cc0435 P 6d39015bc5ef48019f6a3ccff6a9deb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:11.894913 10325 raft_consensus.cc:729] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae43123eb6ab4457925b7c99451b323f, State: Initialized, Role: FOLLOWER
I20260811 17:18:11.894922 10326 raft_consensus.cc:729] T 5e11e667e04040faba80d1d798cc0435 P 6d39015bc5ef48019f6a3ccff6a9deb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d39015bc5ef48019f6a3ccff6a9deb0, State: Initialized, Role: FOLLOWER
I20260811 17:18:11.895419 10325 consensus_queue.cc:260] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d39015bc5ef48019f6a3ccff6a9deb0" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42677 } } peers { permanent_uuid: "dc392b007b9445a5b1db42289c5b56ee" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35379 } } peers { permanent_uuid: "ae43123eb6ab4457925b7c99451b323f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42023 } }
I20260811 17:18:11.895419 10326 consensus_queue.cc:260] T 5e11e667e04040faba80d1d798cc0435 P 6d39015bc5ef48019f6a3ccff6a9deb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d39015bc5ef48019f6a3ccff6a9deb0" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42677 } } peers { permanent_uuid: "dc392b007b9445a5b1db42289c5b56ee" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35379 } } peers { permanent_uuid: "ae43123eb6ab4457925b7c99451b323f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42023 } }
I20260811 17:18:11.896216 10011 heartbeater.cc:503] Master 127.21.54.190:37053 was elected leader, sending a full tablet report...
I20260811 17:18:11.896551 10326 ts_tablet_manager.cc:1428] T 5e11e667e04040faba80d1d798cc0435 P 6d39015bc5ef48019f6a3ccff6a9deb0: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260811 17:18:11.896746 10157 heartbeater.cc:503] Master 127.21.54.190:37053 was elected leader, sending a full tablet report...
I20260811 17:18:11.897807 10325 ts_tablet_manager.cc:1428] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260811 17:18:11.898476 10327 raft_consensus.cc:348] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d39015bc5ef48019f6a3ccff6a9deb0" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42677 } } peers { permanent_uuid: "dc392b007b9445a5b1db42289c5b56ee" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35379 } } peers { permanent_uuid: "ae43123eb6ab4457925b7c99451b323f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42023 } }
I20260811 17:18:11.898825 10327 raft_consensus.cc:374] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:11.898921 10327 raft_consensus.cc:729] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc392b007b9445a5b1db42289c5b56ee, State: Initialized, Role: FOLLOWER
I20260811 17:18:11.899434 10327 consensus_queue.cc:260] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d39015bc5ef48019f6a3ccff6a9deb0" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42677 } } peers { permanent_uuid: "dc392b007b9445a5b1db42289c5b56ee" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35379 } } peers { permanent_uuid: "ae43123eb6ab4457925b7c99451b323f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42023 } }
I20260811 17:18:11.900563 10327 ts_tablet_manager.cc:1428] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260811 17:18:11.901157 10304 heartbeater.cc:503] Master 127.21.54.190:37053 was elected leader, sending a full tablet report...
W20260811 17:18:11.917882 10012 tablet.cc:2369] T 5e11e667e04040faba80d1d798cc0435 P 6d39015bc5ef48019f6a3ccff6a9deb0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:18:11.985872 10332 raft_consensus.cc:482] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:18:11.986166 10332 raft_consensus.cc:504] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d39015bc5ef48019f6a3ccff6a9deb0" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42677 } } peers { permanent_uuid: "dc392b007b9445a5b1db42289c5b56ee" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35379 } } peers { permanent_uuid: "ae43123eb6ab4457925b7c99451b323f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42023 } }
I20260811 17:18:11.987551 10332 leader_election.cc:288] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d39015bc5ef48019f6a3ccff6a9deb0 (127.21.54.129:42677), dc392b007b9445a5b1db42289c5b56ee (127.21.54.131:35379)
I20260811 17:18:11.991912 10159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:11.987889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:42677 (local address 127.21.54.130:43009)
0811 17:18:11.988040 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:11.988057 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:11.988169 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:11.988590 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:11.988596 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:11.988620 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:11.988779 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:11.988788 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.989767 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.989770 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:11.990662 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:11.990672 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.991447 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.991450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:11.991550 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:11.991589 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 17:18:11.991664 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 17:18:11.991738 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 17:18:11.991914 10319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:11.988000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:43009 (local address 127.21.54.129:42677)
0811 17:18:11.988154 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:11.988177 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:11.988202 (+ 25us) server_negotiation.cc:407] Connection header received
0811 17:18:11.988307 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:11.988311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:11.988369 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:11.988580 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:11.988914 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.989644 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.990803 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.991331 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.991427 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:11.991475 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:11.991693 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 17:18:11.991792 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 17:18:11.992386 10335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:11.988223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:35379 (local address 127.21.54.130:57023)
0811 17:18:11.988617 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:11.988633 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:11.988747 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:11.989010 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:11.989013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:11.989033 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:11.989171 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:11.989179 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.990088 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.990091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:11.990940 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:11.990951 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.991689 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.991692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:11.991785 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:11.992053 (+ 268us) client_negotiation.cc:769] Sending connection context
0811 17:18:11.992175 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 17:18:11.992244 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":144,"threads_started":1}
I20260811 17:18:11.992628 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:11.988376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:57023 (local address 127.21.54.131:35379)
0811 17:18:11.988576 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:11.988581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:11.988717 (+ 136us) server_negotiation.cc:407] Connection header received
0811 17:18:11.988833 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:11.988837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:11.988911 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:11.988975 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:11.989321 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.989937 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.991070 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:11.991536 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:11.991662 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:11.991705 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:11.992394 (+ 689us) server_negotiation.cc:299] Negotiation successful
0811 17:18:11.992478 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 17:18:11.992646 9959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e11e667e04040faba80d1d798cc0435" candidate_uuid: "ae43123eb6ab4457925b7c99451b323f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d39015bc5ef48019f6a3ccff6a9deb0" is_pre_election: true
I20260811 17:18:11.993036 9959 raft_consensus.cc:2454] T 5e11e667e04040faba80d1d798cc0435 P 6d39015bc5ef48019f6a3ccff6a9deb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae43123eb6ab4457925b7c99451b323f in term 0.
I20260811 17:18:11.993402 10254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e11e667e04040faba80d1d798cc0435" candidate_uuid: "ae43123eb6ab4457925b7c99451b323f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc392b007b9445a5b1db42289c5b56ee" is_pre_election: true
I20260811 17:18:11.993582 10036 leader_election.cc:302] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d39015bc5ef48019f6a3ccff6a9deb0, ae43123eb6ab4457925b7c99451b323f; no voters:
I20260811 17:18:11.993695 10254 raft_consensus.cc:2454] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae43123eb6ab4457925b7c99451b323f in term 0.
I20260811 17:18:11.993887 10332 raft_consensus.cc:2781] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:18:11.994046 10332 raft_consensus.cc:482] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:18:11.994104 10332 raft_consensus.cc:3037] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:11.995250 10332 raft_consensus.cc:504] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d39015bc5ef48019f6a3ccff6a9deb0" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42677 } } peers { permanent_uuid: "dc392b007b9445a5b1db42289c5b56ee" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35379 } } peers { permanent_uuid: "ae43123eb6ab4457925b7c99451b323f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42023 } }
I20260811 17:18:11.995673 10332 leader_election.cc:288] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [CANDIDATE]: Term 1 election: Requested vote from peers 6d39015bc5ef48019f6a3ccff6a9deb0 (127.21.54.129:42677), dc392b007b9445a5b1db42289c5b56ee (127.21.54.131:35379)
I20260811 17:18:11.996143 9959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e11e667e04040faba80d1d798cc0435" candidate_uuid: "ae43123eb6ab4457925b7c99451b323f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d39015bc5ef48019f6a3ccff6a9deb0"
I20260811 17:18:11.996142 10254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e11e667e04040faba80d1d798cc0435" candidate_uuid: "ae43123eb6ab4457925b7c99451b323f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc392b007b9445a5b1db42289c5b56ee"
I20260811 17:18:11.996287 10254 raft_consensus.cc:3037] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:11.996289 9959 raft_consensus.cc:3037] T 5e11e667e04040faba80d1d798cc0435 P 6d39015bc5ef48019f6a3ccff6a9deb0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:11.997785 10254 raft_consensus.cc:2454] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae43123eb6ab4457925b7c99451b323f in term 1.
I20260811 17:18:11.997785 9959 raft_consensus.cc:2454] T 5e11e667e04040faba80d1d798cc0435 P 6d39015bc5ef48019f6a3ccff6a9deb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae43123eb6ab4457925b7c99451b323f in term 1.
I20260811 17:18:11.998135 10036 leader_election.cc:302] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d39015bc5ef48019f6a3ccff6a9deb0, ae43123eb6ab4457925b7c99451b323f; no voters:
I20260811 17:18:11.998333 10332 raft_consensus.cc:2781] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:11.998586 10332 raft_consensus.cc:686] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [term 1 LEADER]: Becoming Leader. State: Replica: ae43123eb6ab4457925b7c99451b323f, State: Running, Role: LEADER
I20260811 17:18:11.999001 10332 consensus_queue.cc:237] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d39015bc5ef48019f6a3ccff6a9deb0" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42677 } } peers { permanent_uuid: "dc392b007b9445a5b1db42289c5b56ee" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35379 } } peers { permanent_uuid: "ae43123eb6ab4457925b7c99451b323f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42023 } }
I20260811 17:18:12.002703 9802 catalog_manager.cc:5495] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f reported cstate change: term changed from 0 to 1, leader changed from <none> to ae43123eb6ab4457925b7c99451b323f (127.21.54.130). New cstate: current_term: 1 leader_uuid: "ae43123eb6ab4457925b7c99451b323f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d39015bc5ef48019f6a3ccff6a9deb0" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc392b007b9445a5b1db42289c5b56ee" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae43123eb6ab4457925b7c99451b323f" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 42023 } health_report { overall_health: HEALTHY } } }
I20260811 17:18:12.031971 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:12.028138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41104 (local address 127.21.54.130:42023)
0811 17:18:12.028296 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:12.028299 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:12.028312 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:18:12.028387 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:12.028389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:12.028494 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:12.028687 (+ 193us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:18:12.028963 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:12.029489 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:12.030361 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:12.030592 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:12.030705 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:12.031036 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:18:12.031521 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:18:12.031656 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:12.031693 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 17:18:12.031770 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 17:18:12.038488 10254 raft_consensus.cc:1261] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee [term 1 FOLLOWER]: Refusing update from remote peer ae43123eb6ab4457925b7c99451b323f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:18:12.038661 9959 raft_consensus.cc:1261] T 5e11e667e04040faba80d1d798cc0435 P 6d39015bc5ef48019f6a3ccff6a9deb0 [term 1 FOLLOWER]: Refusing update from remote peer ae43123eb6ab4457925b7c99451b323f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:18:12.039462 10336 consensus_queue.cc:1035] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc392b007b9445a5b1db42289c5b56ee" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 35379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:18:12.039659 10332 consensus_queue.cc:1035] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d39015bc5ef48019f6a3ccff6a9deb0" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 42677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:18:12.049331 10342 mvcc.cc:205] Tried to move back new op lower bound from 7317375762580955136 to 7317375762430730240. Current Snapshot: MvccSnapshot[applied={T|T < 7317375762580955136}]
I20260811 17:18:12.058513 9802 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:18:12.058681 9802 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:18:12.061365 9802 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c353cebb7dc45aeafc25ea15516f045: Sending DeleteTablet for 3 replicas of tablet 5e11e667e04040faba80d1d798cc0435
I20260811 17:18:12.062294 10084 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e11e667e04040faba80d1d798cc0435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:18:12 UTC) from {username='slave'} at 127.0.0.1:41090
I20260811 17:18:12.062443 9937 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e11e667e04040faba80d1d798cc0435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:18:12 UTC) from {username='slave'} at 127.0.0.1:34984
I20260811 17:18:12.062474 10233 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e11e667e04040faba80d1d798cc0435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:18:12 UTC) from {username='slave'} at 127.0.0.1:54854
I20260811 17:18:12.063100 10353 tablet_replica.cc:330] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f: stopping tablet replica
I20260811 17:18:12.063285 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 9871
I20260811 17:18:12.064070 10353 raft_consensus.cc:2229] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:18:12.064652 10353 raft_consensus.cc:2258] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:18:12.065682 10354 tablet_replica.cc:330] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee: stopping tablet replica
I20260811 17:18:12.066084 10354 raft_consensus.cc:2229] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:18:12.066504 10354 raft_consensus.cc:2258] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:18:12.068840 10353 ts_tablet_manager.cc:1905] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:18:12.070250 10354 ts_tablet_manager.cc:1905] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:18:12.071944 10353 ts_tablet_manager.cc:1918] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:18:12.072130 10353 log.cc:1196] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/5e11e667e04040faba80d1d798cc0435
I20260811 17:18:12.072571 10353 ts_tablet_manager.cc:1939] T 5e11e667e04040faba80d1d798cc0435 P ae43123eb6ab4457925b7c99451b323f: Deleting consensus metadata
I20260811 17:18:12.073797 10354 ts_tablet_manager.cc:1918] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:18:12.073900 10354 log.cc:1196] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/5e11e667e04040faba80d1d798cc0435
I20260811 17:18:12.073956 9788 catalog_manager.cc:4841] TS ae43123eb6ab4457925b7c99451b323f (127.21.54.130:42023): tablet 5e11e667e04040faba80d1d798cc0435 (table test-table [id=0a17553ec7cc4e0bbfde4d271e35eb5f]) successfully deleted
I20260811 17:18:12.074198 10354 ts_tablet_manager.cc:1939] T 5e11e667e04040faba80d1d798cc0435 P dc392b007b9445a5b1db42289c5b56ee: Deleting consensus metadata
I20260811 17:18:12.075762 9786 catalog_manager.cc:4841] TS dc392b007b9445a5b1db42289c5b56ee (127.21.54.131:35379): tablet 5e11e667e04040faba80d1d798cc0435 (table test-table [id=0a17553ec7cc4e0bbfde4d271e35eb5f]) successfully deleted
W20260811 17:18:12.076210 9786 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.54.129:42677 (error 108)
I20260811 17:18:12.076439 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 10014
I20260811 17:18:12.080219 10321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:12.079913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:42677 (local address 127.0.0.1:34986)
0811 17:18:12.080066 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:12.080143 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.129:42677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 17:18:12.080389 9786 catalog_manager.cc:4584] TS 6d39015bc5ef48019f6a3ccff6a9deb0 (127.21.54.129:42677): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5e11e667e04040faba80d1d798cc0435: Network error: Client connection negotiation failed: client connection to 127.21.54.129:42677: connect: Connection refused (error 111)
I20260811 17:18:12.087029 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 10160
I20260811 17:18:12.098269 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 9770
2026-08-11T17:18:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1161 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:12 dist-test-slave-fd7l krb5kdc[10359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:12 dist-test-slave-fd7l krb5kdc[10359](info): set up 2 sockets
Aug 11 17:18:12 dist-test-slave-fd7l krb5kdc[10359](info): commencing operation
krb5kdc: starting...
W20260811 17:18:13.276751 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:13 dist-test-slave-fd7l krb5kdc[10359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:18:13.415205 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:44125
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:44755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:44125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:13.541041 10376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:13.541343 10376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:13.541389 10376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:13.545634 10376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:13.545702 10376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:13.545722 10376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:13.545787 10376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:13.545825 10376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:13.551383 10376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:44755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:44125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:44125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:13.552536 10376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:13.553574 10376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:13.562963 10386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:13.563053 10383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:13.562974 10384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:13.563040 10376 server_base.cc:1034] running on GCE node
I20260811 17:18:13.564230 10376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:13.565513 10376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:13.566727 10376 hybrid_clock.cc:648] HybridClock initialized: now 1786468693566695 us; error 46 us; skew 500 ppm
Aug 11 17:18:13 dist-test-slave-fd7l krb5kdc[10359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468693, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:13.576957 10376 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:18:13.578179 10376 webserver.cc:460] Webserver started at http://127.21.54.190:34039/ using document root <none> and password file <none>
I20260811 17:18:13.578826 10376 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:13.578929 10376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:13.579196 10376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:13.581529 10376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "00b31469af0b474ea07b01a03d32ce07"
format_stamp: "Formatted at 2026-08-11 17:18:13 on dist-test-slave-fd7l"
server_key: "03b09dd30f13625df49bf2f57a146e4c"
server_key_iv: "6d6e15aece3614493b307e374c8d649b"
server_key_version: "encryptionkey@0"
I20260811 17:18:13.582177 10376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "00b31469af0b474ea07b01a03d32ce07"
format_stamp: "Formatted at 2026-08-11 17:18:13 on dist-test-slave-fd7l"
server_key: "03b09dd30f13625df49bf2f57a146e4c"
server_key_iv: "6d6e15aece3614493b307e374c8d649b"
server_key_version: "encryptionkey@0"
I20260811 17:18:13.586483 10376 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 17:18:13.589450 10393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:13.590955 10376 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 17:18:13.591162 10376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "00b31469af0b474ea07b01a03d32ce07"
format_stamp: "Formatted at 2026-08-11 17:18:13 on dist-test-slave-fd7l"
server_key: "03b09dd30f13625df49bf2f57a146e4c"
server_key_iv: "6d6e15aece3614493b307e374c8d649b"
server_key_version: "encryptionkey@0"
I20260811 17:18:13.591315 10376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:13.613981 10376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:13.617597 10376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:13.617825 10376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:13.625921 10376 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:44125
I20260811 17:18:13.625934 10449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:44125 every 8 connection(s)
I20260811 17:18:13.626957 10376 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:13.630013 10450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:13.634488 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 10376
I20260811 17:18:13.634682 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:13.635006 21722 external_mini_cluster.cc:1428] Setting key 299ab7f925394877deb1d8df503e4466
I20260811 17:18:13.636760 10450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07: Bootstrap starting.
I20260811 17:18:13.639249 10450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:13.640079 10450 log.cc:824] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:13.642198 10450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07: No bootstrap required, opened a new log
Aug 11 17:18:13 dist-test-slave-fd7l krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:18:13.647063 10450 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b31469af0b474ea07b01a03d32ce07" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44125 } }
I20260811 17:18:13.647320 10450 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:13.647387 10450 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00b31469af0b474ea07b01a03d32ce07, State: Initialized, Role: FOLLOWER
I20260811 17:18:13.647954 10450 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b31469af0b474ea07b01a03d32ce07" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44125 } }
I20260811 17:18:13.648171 10450 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:13.648285 10450 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:13.648404 10450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:13.649520 10450 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b31469af0b474ea07b01a03d32ce07" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44125 } }
I20260811 17:18:13.649945 10450 leader_election.cc:302] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00b31469af0b474ea07b01a03d32ce07; no voters:
I20260811 17:18:13.650331 10450 leader_election.cc:288] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:13.650441 10456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:13.650739 10456 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07 [term 1 LEADER]: Becoming Leader. State: Replica: 00b31469af0b474ea07b01a03d32ce07, State: Running, Role: LEADER
I20260811 17:18:13.651105 10456 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b31469af0b474ea07b01a03d32ce07" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44125 } }
I20260811 17:18:13.651340 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:13.636811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53906 (local address 127.21.54.190:44125)
0811 17:18:13.637331 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:13.637348 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:13.637385 (+ 37us) server_negotiation.cc:407] Connection header received
0811 17:18:13.638044 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:13.638061 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:13.638421 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:13.638799 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:13.639721 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:13.640997 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:13.641883 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:13.642458 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:13.642631 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:13.645562 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:13.645567 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:13.645582 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:13.645613 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:13.647604 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:13.648197 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:13.648203 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:13.648210 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:13.648406 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:13.648802 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:13.648806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:13.648808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:13.649264 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:13.649487 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:13.649796 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 17:18:13.650140 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":156,"threads_started":1}
I20260811 17:18:13.651840 10450 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:13.652379 10456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00b31469af0b474ea07b01a03d32ce07. Latest consensus state: current_term: 1 leader_uuid: "00b31469af0b474ea07b01a03d32ce07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b31469af0b474ea07b01a03d32ce07" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44125 } } }
I20260811 17:18:13.652499 10456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:13.652896 10457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00b31469af0b474ea07b01a03d32ce07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b31469af0b474ea07b01a03d32ce07" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44125 } } }
I20260811 17:18:13.653013 10457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:13.653622 10460 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:13.656903 10460 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:13.663542 10460 catalog_manager.cc:1347] Generated new cluster ID: da3ba7ea953d43c7bc06ef2a371d2f84
I20260811 17:18:13.663655 10460 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:13.671392 10460 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:13.672647 10460 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:13.678682 10460 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07: Generated new TSK 0
I20260811 17:18:13.679554 10460 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
I20260811 17:18:13.723668 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44125
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:44755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:13.870441 10479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:13.870748 10479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:13.870795 10479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:13.875144 10479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:13.875226 10479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:13.875288 10479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:18:13.881078 10479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:44755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:13.882251 10479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:13.883359 10479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:13.894097 10487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:13.894097 10484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:13.894356 10485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:13.894485 10479 server_base.cc:1034] running on GCE node
I20260811 17:18:13.894990 10479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:13.895680 10479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:13.896934 10479 hybrid_clock.cc:648] HybridClock initialized: now 1786468693896911 us; error 30 us; skew 500 ppm
Aug 11 17:18:13 dist-test-slave-fd7l krb5kdc[10359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468693, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:13.900213 10479 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:18:13.901366 10479 webserver.cc:460] Webserver started at http://127.21.54.129:36051/ using document root <none> and password file <none>
I20260811 17:18:13.901993 10479 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:13.902088 10479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:13.902359 10479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:13.904666 10479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "249a0df663e74caf897202149a7da5eb"
format_stamp: "Formatted at 2026-08-11 17:18:13 on dist-test-slave-fd7l"
server_key: "9c72e80d8565562d33ec9c7ac5c40b66"
server_key_iv: "9db1c0b0d87b791077bb7065010f657a"
server_key_version: "encryptionkey@0"
I20260811 17:18:13.905270 10479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "249a0df663e74caf897202149a7da5eb"
format_stamp: "Formatted at 2026-08-11 17:18:13 on dist-test-slave-fd7l"
server_key: "9c72e80d8565562d33ec9c7ac5c40b66"
server_key_iv: "9db1c0b0d87b791077bb7065010f657a"
server_key_version: "encryptionkey@0"
I20260811 17:18:13.909372 10479 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:18:13.912218 10495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:13.913766 10479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:13.913923 10479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "249a0df663e74caf897202149a7da5eb"
format_stamp: "Formatted at 2026-08-11 17:18:13 on dist-test-slave-fd7l"
server_key: "9c72e80d8565562d33ec9c7ac5c40b66"
server_key_iv: "9db1c0b0d87b791077bb7065010f657a"
server_key_version: "encryptionkey@0"
I20260811 17:18:13.914064 10479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:13.943066 10479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:13.946741 10479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:13.946965 10479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:13.947621 10479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:18:13.948581 10479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:18:13.948652 10479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:13.948726 10479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:18:13.948768 10479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:13.958874 10479 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:38235
I20260811 17:18:13.958892 10626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:38235 every 8 connection(s)
I20260811 17:18:13.959872 10479 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:18:13.963984 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 10479
I20260811 17:18:13.964140 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:18:13.964426 21722 external_mini_cluster.cc:1428] Setting key b658c227af4f7c0719c6b650efee214c
Aug 11 17:18:13 dist-test-slave-fd7l krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468693, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:18:13.975430 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:13.961995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:59341 (local address 127.21.54.190:44125)
0811 17:18:13.962193 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:13.962197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:13.963127 (+ 930us) server_negotiation.cc:407] Connection header received
0811 17:18:13.964660 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:13.964664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:13.964727 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:13.964831 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:13.966258 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:13.967091 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:13.968467 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:13.968759 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:13.968844 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:13.971657 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:13.971661 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:13.971662 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:13.971688 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:13.973224 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:13.973795 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:13.973799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:13.973800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:13.973887 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:13.974258 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:13.974261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:13.974263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:13.974456 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:13.974579 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:13.975114 (+ 535us) server_negotiation.cc:299] Negotiation successful
0811 17:18:13.975257 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
WARNING: no policy specified for kudu/127.21.54.130@KRBTEST.COM; defaulting to no policy
I20260811 17:18:13.976390 10630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:13.962446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:44125 (local address 127.21.54.129:59341)
0811 17:18:13.962932 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:13.962978 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:13.964376 (+ 1398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:13.965093 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:13.965111 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:13.965479 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:13.966076 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:13.966098 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:13.967231 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:13.967233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:13.968319 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:13.968334 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:13.968926 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:13.968932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:13.969198 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:13.969877 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:13.969920 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:13.971458 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:13.973356 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:13.973362 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:13.973381 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:13.973683 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:13.973984 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:13.973988 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:13.973990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:13.974156 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:13.974571 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:13.974578 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:13.974864 (+ 286us) client_negotiation.cc:769] Sending connection context
0811 17:18:13.975078 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 17:18:13.975436 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":125,"threads_started":1}
I20260811 17:18:13.978127 10627 heartbeater.cc:347] Connected to a master server at 127.21.54.190:44125
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.130@KRBTEST.COM" created.
I20260811 17:18:13.978461 10627 heartbeater.cc:464] Registering TS with master...
I20260811 17:18:13.979142 10627 heartbeater.cc:511] Master 127.21.54.190:44125 requested a full tablet report, sending...
I20260811 17:18:13.981097 10409 ts_manager.cc:194] Registered new tserver with Master: 249a0df663e74caf897202149a7da5eb (127.21.54.129:38235)
I20260811 17:18:13.982566 10409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:59341
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
I20260811 17:18:14.000898 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44125
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:44755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:14.120911 10633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:14.121256 10633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:14.121354 10633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:14.125809 10633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:14.125910 10633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:14.126024 10633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:18:14.131616 10633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:44755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:14.132927 10633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:14.134048 10633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:18:14.144407 10633 server_base.cc:1034] running on GCE node
W20260811 17:18:14.144366 10639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:14.144375 10638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:14.144366 10641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:14.145103 10633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:14.145792 10633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:14.147147 10633 hybrid_clock.cc:648] HybridClock initialized: now 1786468694147124 us; error 30 us; skew 500 ppm
Aug 11 17:18:14 dist-test-slave-fd7l krb5kdc[10359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468694, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:14.150344 10633 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:18:14.151286 10633 webserver.cc:460] Webserver started at http://127.21.54.130:42185/ using document root <none> and password file <none>
I20260811 17:18:14.151877 10633 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:14.151942 10633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:14.152184 10633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:14.154548 10633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "80fc770454ad46b78d16edbc0c470d1b"
format_stamp: "Formatted at 2026-08-11 17:18:14 on dist-test-slave-fd7l"
server_key: "779065b13004bd9d272b87e2a70c2fb6"
server_key_iv: "b4b100b70b3f6ea7f3be1c4dc07e1095"
server_key_version: "encryptionkey@0"
I20260811 17:18:14.155090 10633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "80fc770454ad46b78d16edbc0c470d1b"
format_stamp: "Formatted at 2026-08-11 17:18:14 on dist-test-slave-fd7l"
server_key: "779065b13004bd9d272b87e2a70c2fb6"
server_key_iv: "b4b100b70b3f6ea7f3be1c4dc07e1095"
server_key_version: "encryptionkey@0"
I20260811 17:18:14.159162 10633 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:18:14.161863 10648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:14.163264 10633 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:18:14.163367 10633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "80fc770454ad46b78d16edbc0c470d1b"
format_stamp: "Formatted at 2026-08-11 17:18:14 on dist-test-slave-fd7l"
server_key: "779065b13004bd9d272b87e2a70c2fb6"
server_key_iv: "b4b100b70b3f6ea7f3be1c4dc07e1095"
server_key_version: "encryptionkey@0"
I20260811 17:18:14.163465 10633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:14.193123 10633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:14.196439 10633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:14.196671 10633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:14.197258 10633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:18:14.198088 10633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:18:14.198136 10633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:14.198176 10633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:18:14.198192 10633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:14.207221 10633 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:45697
I20260811 17:18:14.207242 10776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:45697 every 8 connection(s)
I20260811 17:18:14.208159 10633 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:18:14.209312 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 10633
I20260811 17:18:14.209442 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:18:14.209772 21722 external_mini_cluster.cc:1428] Setting key 5dba4f9b1a2e97b70d01adc88d26059c
Aug 11 17:18:14 dist-test-slave-fd7l krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468694, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.54.131@KRBTEST.COM; defaulting to no policy
I20260811 17:18:14.224412 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:14.210265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:60291 (local address 127.21.54.190:44125)
0811 17:18:14.210409 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:14.210416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:14.211982 (+ 1566us) server_negotiation.cc:407] Connection header received
0811 17:18:14.213020 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:14.213027 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:14.213094 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:14.213202 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:14.214916 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.215442 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.216754 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.216996 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.217113 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:14.220265 (+ 3152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:14.220268 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:14.220270 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:14.220299 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:14.222008 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:14.222609 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:14.222612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:14.222614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:14.222703 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:14.223134 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:14.223140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:14.223142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:14.223371 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:14.223465 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:14.224020 (+ 555us) server_negotiation.cc:299] Negotiation successful
0811 17:18:14.224144 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.131@KRBTEST.COM" created.
I20260811 17:18:14.225469 10779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:14.210748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:44125 (local address 127.21.54.130:60291)
0811 17:18:14.211797 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:14.211854 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:14.212803 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:14.213448 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:14.213459 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:14.213985 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:14.214731 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:14.214755 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.215581 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.215584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:14.216615 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:14.216625 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.217176 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.217181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:14.217360 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:14.218043 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:14.218061 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:14.220040 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:14.222186 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:14.222192 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:14.222208 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:14.222482 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:14.222796 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:14.222798 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:14.222800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:14.222989 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:14.223484 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:14.223489 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:14.223780 (+ 291us) client_negotiation.cc:769] Sending connection context
0811 17:18:14.223954 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 17:18:14.224244 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"thread_start_us":671,"threads_started":1}
I20260811 17:18:14.226905 10777 heartbeater.cc:347] Connected to a master server at 127.21.54.190:44125
I20260811 17:18:14.227175 10777 heartbeater.cc:464] Registering TS with master...
I20260811 17:18:14.227715 10777 heartbeater.cc:511] Master 127.21.54.190:44125 requested a full tablet report, sending...
I20260811 17:18:14.228976 10409 ts_manager.cc:194] Registered new tserver with Master: 80fc770454ad46b78d16edbc0c470d1b (127.21.54.130:45697)
I20260811 17:18:14.229903 10409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:60291
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
I20260811 17:18:14.244992 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44125
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:44755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:14.364744 10782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:14.365069 10782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:14.365145 10782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:14.369439 10782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:14.369592 10782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:14.369683 10782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:18:14.375483 10782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:44755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:14.376720 10782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:14.377813 10782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:14.388578 10789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:14.388626 10791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:14.388618 10788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:14.388867 10782 server_base.cc:1034] running on GCE node
I20260811 17:18:14.389410 10782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:14.390102 10782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:14.391299 10782 hybrid_clock.cc:648] HybridClock initialized: now 1786468694391284 us; error 30 us; skew 500 ppm
Aug 11 17:18:14 dist-test-slave-fd7l krb5kdc[10359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468694, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:14.394642 10782 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:18:14.395798 10782 webserver.cc:460] Webserver started at http://127.21.54.131:45167/ using document root <none> and password file <none>
I20260811 17:18:14.396432 10782 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:14.396508 10782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:14.396806 10782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:14.399380 10782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "8298fac5683e46ca9e95192258fcc73f"
format_stamp: "Formatted at 2026-08-11 17:18:14 on dist-test-slave-fd7l"
server_key: "0a995242242c989acffbb60ecb999179"
server_key_iv: "c904188fbfa501517a2addb455c391e0"
server_key_version: "encryptionkey@0"
I20260811 17:18:14.400130 10782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "8298fac5683e46ca9e95192258fcc73f"
format_stamp: "Formatted at 2026-08-11 17:18:14 on dist-test-slave-fd7l"
server_key: "0a995242242c989acffbb60ecb999179"
server_key_iv: "c904188fbfa501517a2addb455c391e0"
server_key_version: "encryptionkey@0"
I20260811 17:18:14.405045 10782 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260811 17:18:14.407891 10798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:14.409462 10782 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:14.409667 10782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "8298fac5683e46ca9e95192258fcc73f"
format_stamp: "Formatted at 2026-08-11 17:18:14 on dist-test-slave-fd7l"
server_key: "0a995242242c989acffbb60ecb999179"
server_key_iv: "c904188fbfa501517a2addb455c391e0"
server_key_version: "encryptionkey@0"
I20260811 17:18:14.409812 10782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:14.423674 10782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:14.427209 10782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:14.427448 10782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:14.428117 10782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:18:14.429105 10782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:18:14.429181 10782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:14.429255 10782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:18:14.429306 10782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:14.438532 10782 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:40873
I20260811 17:18:14.438558 10927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:40873 every 8 connection(s)
I20260811 17:18:14.439508 10782 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:18:14.443876 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 10782
I20260811 17:18:14.444048 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:18:14.444361 21722 external_mini_cluster.cc:1428] Setting key 20b378680e06b2b0e5d19c24e1b3bb53
Aug 11 17:18:14 dist-test-slave-fd7l krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468694, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:18:14.454818 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:14.441768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:56941 (local address 127.21.54.190:44125)
0811 17:18:14.441914 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:14.441918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:14.442809 (+ 891us) server_negotiation.cc:407] Connection header received
0811 17:18:14.444243 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:14.444246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:14.444308 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:14.444419 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:14.446028 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.446844 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.448110 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.448352 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.448569 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:14.451077 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:14.451080 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:14.451082 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:14.451109 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:14.452659 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:14.453238 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:14.453240 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:14.453242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:14.453369 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:14.453728 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:14.453730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:14.453731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:14.453914 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:14.454007 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:14.454576 (+ 569us) server_negotiation.cc:299] Negotiation successful
0811 17:18:14.454690 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 17:18:14.455736 10930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:14.442154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:44125 (local address 127.21.54.131:56941)
0811 17:18:14.442644 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:14.442689 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:14.443973 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:14.444645 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:14.444653 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:14.445072 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:14.445808 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:14.445829 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.447001 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.447004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:14.447954 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:14.447965 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.448494 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.448497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:14.448731 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:14.449306 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:14.449323 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:14.450894 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:14.452774 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:14.452779 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:14.452793 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:14.453142 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:14.453444 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:14.453446 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:14.453448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:14.453642 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:14.454025 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:14.454030 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:14.454289 (+ 259us) client_negotiation.cc:769] Sending connection context
0811 17:18:14.454518 (+ 229us) client_negotiation.cc:240] Negotiation successful
0811 17:18:14.454852 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":117,"threads_started":1}
I20260811 17:18:14.457005 10928 heartbeater.cc:347] Connected to a master server at 127.21.54.190:44125
I20260811 17:18:14.457257 10928 heartbeater.cc:464] Registering TS with master...
I20260811 17:18:14.457901 10928 heartbeater.cc:511] Master 127.21.54.190:44125 requested a full tablet report, sending...
I20260811 17:18:14.459100 10409 ts_manager.cc:194] Registered new tserver with Master: 8298fac5683e46ca9e95192258fcc73f (127.21.54.131:40873)
I20260811 17:18:14.459666 10409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:56941
I20260811 17:18:14.471575 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:18:14.482466 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:14.473810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53918 (local address 127.21.54.190:44125)
0811 17:18:14.473968 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:14.473982 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:14.474132 (+ 150us) server_negotiation.cc:407] Connection header received
0811 17:18:14.474223 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:14.474225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:14.474272 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:14.474349 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:14.475225 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.475875 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.476946 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.477181 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.477275 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:14.479102 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:14.479105 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:14.479107 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:14.479136 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:14.480482 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:14.481064 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:14.481067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:14.481068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:14.481152 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:14.481540 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:14.481543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:14.481544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:14.481780 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:14.481924 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:14.482233 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 17:18:14.482338 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 17:18:14.485952 10409 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:18:14.489359 10409 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:18:14.504953 10947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:14.499632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35716 (local address 127.21.54.131:40873)
0811 17:18:14.500202 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:14.500217 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:14.500242 (+ 25us) server_negotiation.cc:407] Connection header received
0811 17:18:14.500312 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:14.500317 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:14.500484 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:14.500667 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:14.501022 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.502096 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.503522 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.504101 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.504227 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:14.504391 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:14.504676 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 17:18:14.504791 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":166,"threads_started":1}
I20260811 17:18:14.504956 10942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:14.499030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:40873 (local address 127.0.0.1:35716)
0811 17:18:14.499885 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:14.499899 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:14.499995 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:14.500639 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:14.500645 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:14.500669 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:14.500913 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:14.500918 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.502269 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.502272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:14.503406 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:14.503414 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.504232 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.504235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:14.504407 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:14.504562 (+ 155us) client_negotiation.cc:769] Sending connection context
0811 17:18:14.504640 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 17:18:14.504799 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"mutex_wait_us":154,"thread_start_us":136,"threads_started":1}
I20260811 17:18:14.504972 10941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:14.498388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:45697 (local address 127.0.0.1:57466)
0811 17:18:14.499235 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:14.499290 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:14.499436 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:14.500718 (+ 1282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:14.500722 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:14.500735 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:14.500822 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:14.500828 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.501872 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.501875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:14.503211 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:14.503222 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.504252 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.504256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:14.504445 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:14.504524 (+ 79us) client_negotiation.cc:769] Sending connection context
0811 17:18:14.504630 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 17:18:14.504776 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":66,"thread_start_us":85,"threads_started":1}
I20260811 17:18:14.505286 10943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:14.498544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53460 (local address 127.21.54.129:38235)
0811 17:18:14.498849 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:14.498857 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:14.499605 (+ 748us) server_negotiation.cc:407] Connection header received
0811 17:18:14.499688 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:14.499692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:14.499921 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:14.500095 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:14.500481 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.501638 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.503240 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.504108 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.504227 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:14.504391 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:14.505051 (+ 660us) server_negotiation.cc:299] Negotiation successful
0811 17:18:14.505154 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":123,"threads_started":1}
I20260811 17:18:14.505618 10946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:14.499240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57466 (local address 127.21.54.130:45697)
0811 17:18:14.500241 (+ 1001us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:14.500253 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:14.500275 (+ 22us) server_negotiation.cc:407] Connection header received
0811 17:18:14.500346 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:14.500351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:14.500572 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:14.500829 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:14.501003 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.501738 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.503318 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.504108 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.505146 (+ 1038us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:14.505281 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:14.505337 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 17:18:14.505437 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":904,"thread_start_us":119,"threads_started":1}
I20260811 17:18:14.504982 10945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:14.498388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:38235 (local address 127.0.0.1:53460)
0811 17:18:14.499404 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:14.499417 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:14.499502 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:14.500156 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:14.500163 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:14.500189 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:14.500350 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:14.500358 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.501769 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.501773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:14.503103 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:14.503113 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.504451 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.504454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:14.504619 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:14.504644 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 17:18:14.504705 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 17:18:14.504776 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":948,"mutex_wait_us":93,"thread_start_us":95,"threads_started":1}
I20260811 17:18:14.508337 10552 tablet_service.cc:1467] Processing CreateTablet for tablet 0f65c909ba2c4544ada86e4bf3917419 (DEFAULT_TABLE table=test-table [id=7bffd736703247248bc85e002601fc18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:18:14.508488 10705 tablet_service.cc:1467] Processing CreateTablet for tablet 0f65c909ba2c4544ada86e4bf3917419 (DEFAULT_TABLE table=test-table [id=7bffd736703247248bc85e002601fc18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:18:14.508486 10852 tablet_service.cc:1467] Processing CreateTablet for tablet 0f65c909ba2c4544ada86e4bf3917419 (DEFAULT_TABLE table=test-table [id=7bffd736703247248bc85e002601fc18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:18:14.509485 10552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f65c909ba2c4544ada86e4bf3917419. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:14.509529 10705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f65c909ba2c4544ada86e4bf3917419. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:14.509528 10852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f65c909ba2c4544ada86e4bf3917419. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:14.516079 10948 tablet_bootstrap.cc:492] T 0f65c909ba2c4544ada86e4bf3917419 P 249a0df663e74caf897202149a7da5eb: Bootstrap starting.
I20260811 17:18:14.518083 10949 tablet_bootstrap.cc:492] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f: Bootstrap starting.
I20260811 17:18:14.518625 10948 tablet_bootstrap.cc:654] T 0f65c909ba2c4544ada86e4bf3917419 P 249a0df663e74caf897202149a7da5eb: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:14.519464 10948 log.cc:824] T 0f65c909ba2c4544ada86e4bf3917419 P 249a0df663e74caf897202149a7da5eb: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:14.520148 10949 tablet_bootstrap.cc:654] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:14.520988 10950 tablet_bootstrap.cc:492] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b: Bootstrap starting.
I20260811 17:18:14.521502 10949 log.cc:824] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:14.521879 10948 tablet_bootstrap.cc:492] T 0f65c909ba2c4544ada86e4bf3917419 P 249a0df663e74caf897202149a7da5eb: No bootstrap required, opened a new log
I20260811 17:18:14.522130 10948 ts_tablet_manager.cc:1397] T 0f65c909ba2c4544ada86e4bf3917419 P 249a0df663e74caf897202149a7da5eb: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260811 17:18:14.523450 10950 tablet_bootstrap.cc:654] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:14.523870 10949 tablet_bootstrap.cc:492] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f: No bootstrap required, opened a new log
I20260811 17:18:14.524104 10949 ts_tablet_manager.cc:1397] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 17:18:14.524806 10950 log.cc:824] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:14.526206 10948 raft_consensus.cc:348] T 0f65c909ba2c4544ada86e4bf3917419 P 249a0df663e74caf897202149a7da5eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "249a0df663e74caf897202149a7da5eb" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38235 } } peers { permanent_uuid: "80fc770454ad46b78d16edbc0c470d1b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45697 } } peers { permanent_uuid: "8298fac5683e46ca9e95192258fcc73f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40873 } }
I20260811 17:18:14.526474 10948 raft_consensus.cc:374] T 0f65c909ba2c4544ada86e4bf3917419 P 249a0df663e74caf897202149a7da5eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:14.526557 10948 raft_consensus.cc:729] T 0f65c909ba2c4544ada86e4bf3917419 P 249a0df663e74caf897202149a7da5eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 249a0df663e74caf897202149a7da5eb, State: Initialized, Role: FOLLOWER
I20260811 17:18:14.526966 10950 tablet_bootstrap.cc:492] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b: No bootstrap required, opened a new log
I20260811 17:18:14.527072 10948 consensus_queue.cc:260] T 0f65c909ba2c4544ada86e4bf3917419 P 249a0df663e74caf897202149a7da5eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "249a0df663e74caf897202149a7da5eb" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38235 } } peers { permanent_uuid: "80fc770454ad46b78d16edbc0c470d1b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45697 } } peers { permanent_uuid: "8298fac5683e46ca9e95192258fcc73f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40873 } }
I20260811 17:18:14.527236 10950 ts_tablet_manager.cc:1397] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260811 17:18:14.527959 10949 raft_consensus.cc:348] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "249a0df663e74caf897202149a7da5eb" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38235 } } peers { permanent_uuid: "80fc770454ad46b78d16edbc0c470d1b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45697 } } peers { permanent_uuid: "8298fac5683e46ca9e95192258fcc73f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40873 } }
I20260811 17:18:14.528241 10949 raft_consensus.cc:374] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:14.528278 10948 ts_tablet_manager.cc:1428] T 0f65c909ba2c4544ada86e4bf3917419 P 249a0df663e74caf897202149a7da5eb: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 17:18:14.528357 10949 raft_consensus.cc:729] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8298fac5683e46ca9e95192258fcc73f, State: Initialized, Role: FOLLOWER
I20260811 17:18:14.528697 10627 heartbeater.cc:503] Master 127.21.54.190:44125 was elected leader, sending a full tablet report...
I20260811 17:18:14.528846 10949 consensus_queue.cc:260] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "249a0df663e74caf897202149a7da5eb" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38235 } } peers { permanent_uuid: "80fc770454ad46b78d16edbc0c470d1b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45697 } } peers { permanent_uuid: "8298fac5683e46ca9e95192258fcc73f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40873 } }
I20260811 17:18:14.529534 10928 heartbeater.cc:503] Master 127.21.54.190:44125 was elected leader, sending a full tablet report...
I20260811 17:18:14.529830 10949 ts_tablet_manager.cc:1428] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 17:18:14.531117 10950 raft_consensus.cc:348] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "249a0df663e74caf897202149a7da5eb" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38235 } } peers { permanent_uuid: "80fc770454ad46b78d16edbc0c470d1b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45697 } } peers { permanent_uuid: "8298fac5683e46ca9e95192258fcc73f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40873 } }
I20260811 17:18:14.531397 10950 raft_consensus.cc:374] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:14.531497 10950 raft_consensus.cc:729] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80fc770454ad46b78d16edbc0c470d1b, State: Initialized, Role: FOLLOWER
I20260811 17:18:14.532055 10950 consensus_queue.cc:260] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "249a0df663e74caf897202149a7da5eb" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38235 } } peers { permanent_uuid: "80fc770454ad46b78d16edbc0c470d1b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45697 } } peers { permanent_uuid: "8298fac5683e46ca9e95192258fcc73f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40873 } }
I20260811 17:18:14.532426 10956 raft_consensus.cc:482] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:18:14.532732 10956 raft_consensus.cc:504] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "249a0df663e74caf897202149a7da5eb" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38235 } } peers { permanent_uuid: "80fc770454ad46b78d16edbc0c470d1b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45697 } } peers { permanent_uuid: "8298fac5683e46ca9e95192258fcc73f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40873 } }
I20260811 17:18:14.532829 10777 heartbeater.cc:503] Master 127.21.54.190:44125 was elected leader, sending a full tablet report...
I20260811 17:18:14.533200 10950 ts_tablet_manager.cc:1428] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 17:18:14.534531 10956 leader_election.cc:288] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 249a0df663e74caf897202149a7da5eb (127.21.54.129:38235), 8298fac5683e46ca9e95192258fcc73f (127.21.54.131:40873)
I20260811 17:18:14.538887 10779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:14.534955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:38235 (local address 127.21.54.130:59821)
0811 17:18:14.535064 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:14.535083 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:14.535176 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:14.535536 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:14.535540 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:14.535558 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:14.535680 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:14.535688 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.536637 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.536641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:14.537603 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:14.537612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.538522 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.538525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:14.538629 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:14.538651 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 17:18:14.538717 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 17:18:14.538784 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":3}
I20260811 17:18:14.538952 10943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:14.535083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:59821 (local address 127.21.54.129:38235)
0811 17:18:14.535315 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:14.535322 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:14.535341 (+ 19us) server_negotiation.cc:407] Connection header received
0811 17:18:14.535386 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:14.535389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:14.535437 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:14.535535 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:14.535811 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.536485 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.537728 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.538401 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.538526 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:14.538578 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:14.538756 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 17:18:14.538835 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260811 17:18:14.539548 10576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f65c909ba2c4544ada86e4bf3917419" candidate_uuid: "80fc770454ad46b78d16edbc0c470d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "249a0df663e74caf897202149a7da5eb" is_pre_election: true
I20260811 17:18:14.539906 10576 raft_consensus.cc:2454] T 0f65c909ba2c4544ada86e4bf3917419 P 249a0df663e74caf897202149a7da5eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80fc770454ad46b78d16edbc0c470d1b in term 0.
I20260811 17:18:14.540268 10947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:14.535791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:35169 (local address 127.21.54.131:40873)
0811 17:18:14.535927 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:14.535931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:14.536339 (+ 408us) server_negotiation.cc:407] Connection header received
0811 17:18:14.536569 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:14.536572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:14.536625 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:14.536688 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:14.537182 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.537794 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.539322 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.539750 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.539876 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:14.539914 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:14.540112 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 17:18:14.540180 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 17:18:14.540331 10958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:14.535239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:40873 (local address 127.21.54.130:35169)
0811 17:18:14.536256 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:14.536270 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:14.536407 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:14.536864 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:14.536867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:14.536885 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:14.537044 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:14.537050 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.537933 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.537936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:14.539078 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:14.539091 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.539876 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.539880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:14.539991 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:14.540015 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 17:18:14.540099 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 17:18:14.540166 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":947,"mutex_wait_us":1,"thread_start_us":133,"threads_started":1}
I20260811 17:18:14.540468 10651 leader_election.cc:302] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 249a0df663e74caf897202149a7da5eb, 80fc770454ad46b78d16edbc0c470d1b; no voters:
I20260811 17:18:14.540819 10956 raft_consensus.cc:2781] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:18:14.540916 10956 raft_consensus.cc:482] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:18:14.540964 10956 raft_consensus.cc:3037] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:14.541070 10874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f65c909ba2c4544ada86e4bf3917419" candidate_uuid: "80fc770454ad46b78d16edbc0c470d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8298fac5683e46ca9e95192258fcc73f" is_pre_election: true
I20260811 17:18:14.541338 10874 raft_consensus.cc:2454] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80fc770454ad46b78d16edbc0c470d1b in term 0.
I20260811 17:18:14.542403 10956 raft_consensus.cc:504] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "249a0df663e74caf897202149a7da5eb" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38235 } } peers { permanent_uuid: "80fc770454ad46b78d16edbc0c470d1b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45697 } } peers { permanent_uuid: "8298fac5683e46ca9e95192258fcc73f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40873 } }
I20260811 17:18:14.542856 10956 leader_election.cc:288] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [CANDIDATE]: Term 1 election: Requested vote from peers 249a0df663e74caf897202149a7da5eb (127.21.54.129:38235), 8298fac5683e46ca9e95192258fcc73f (127.21.54.131:40873)
I20260811 17:18:14.543238 10576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f65c909ba2c4544ada86e4bf3917419" candidate_uuid: "80fc770454ad46b78d16edbc0c470d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "249a0df663e74caf897202149a7da5eb"
I20260811 17:18:14.543409 10576 raft_consensus.cc:3037] T 0f65c909ba2c4544ada86e4bf3917419 P 249a0df663e74caf897202149a7da5eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:14.544560 10576 raft_consensus.cc:2454] T 0f65c909ba2c4544ada86e4bf3917419 P 249a0df663e74caf897202149a7da5eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80fc770454ad46b78d16edbc0c470d1b in term 1.
I20260811 17:18:14.544844 10874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f65c909ba2c4544ada86e4bf3917419" candidate_uuid: "80fc770454ad46b78d16edbc0c470d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8298fac5683e46ca9e95192258fcc73f"
I20260811 17:18:14.544890 10651 leader_election.cc:302] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 249a0df663e74caf897202149a7da5eb, 80fc770454ad46b78d16edbc0c470d1b; no voters:
I20260811 17:18:14.544978 10874 raft_consensus.cc:3037] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:14.545187 10956 raft_consensus.cc:2781] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:14.545344 10956 raft_consensus.cc:686] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [term 1 LEADER]: Becoming Leader. State: Replica: 80fc770454ad46b78d16edbc0c470d1b, State: Running, Role: LEADER
I20260811 17:18:14.545765 10956 consensus_queue.cc:237] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "249a0df663e74caf897202149a7da5eb" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38235 } } peers { permanent_uuid: "80fc770454ad46b78d16edbc0c470d1b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45697 } } peers { permanent_uuid: "8298fac5683e46ca9e95192258fcc73f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40873 } }
I20260811 17:18:14.546232 10874 raft_consensus.cc:2454] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80fc770454ad46b78d16edbc0c470d1b in term 1.
I20260811 17:18:14.549336 10409 catalog_manager.cc:5495] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b reported cstate change: term changed from 0 to 1, leader changed from <none> to 80fc770454ad46b78d16edbc0c470d1b (127.21.54.130). New cstate: current_term: 1 leader_uuid: "80fc770454ad46b78d16edbc0c470d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "249a0df663e74caf897202149a7da5eb" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80fc770454ad46b78d16edbc0c470d1b" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 45697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8298fac5683e46ca9e95192258fcc73f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40873 } health_report { overall_health: UNKNOWN } } }
I20260811 17:18:14.564849 10946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:14.560926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57476 (local address 127.21.54.130:45697)
0811 17:18:14.561088 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:14.561091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:14.561105 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:18:14.561176 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:14.561178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:14.561232 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:14.561325 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:18:14.561666 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.562264 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.563223 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.563440 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.563586 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:14.563937 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:18:14.564370 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:18:14.564482 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:14.564674 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 17:18:14.564714 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 17:18:14.571398 10576 raft_consensus.cc:1261] T 0f65c909ba2c4544ada86e4bf3917419 P 249a0df663e74caf897202149a7da5eb [term 1 FOLLOWER]: Refusing update from remote peer 80fc770454ad46b78d16edbc0c470d1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:18:14.571405 10874 raft_consensus.cc:1261] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f [term 1 FOLLOWER]: Refusing update from remote peer 80fc770454ad46b78d16edbc0c470d1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:18:14.572185 10957 consensus_queue.cc:1035] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8298fac5683e46ca9e95192258fcc73f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 40873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:18:14.572404 10956 consensus_queue.cc:1035] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "249a0df663e74caf897202149a7da5eb" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 38235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:18:14.581547 10965 mvcc.cc:205] Tried to move back new op lower bound from 7317375772955836416 to 7317375772862636032. Current Snapshot: MvccSnapshot[applied={T|T < 7317375772955836416}]
I20260811 17:18:14.592883 10409 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:18:14.593114 10409 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:18:14.596369 10409 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00b31469af0b474ea07b01a03d32ce07: Sending DeleteTablet for 3 replicas of tablet 0f65c909ba2c4544ada86e4bf3917419
I20260811 17:18:14.597375 10705 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f65c909ba2c4544ada86e4bf3917419 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:18:14 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:57466
I20260811 17:18:14.597386 10552 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f65c909ba2c4544ada86e4bf3917419 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:18:14 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:53460
I20260811 17:18:14.597826 10852 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f65c909ba2c4544ada86e4bf3917419 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:18:14 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:35716
I20260811 17:18:14.598021 10974 tablet_replica.cc:330] T 0f65c909ba2c4544ada86e4bf3917419 P 249a0df663e74caf897202149a7da5eb: stopping tablet replica
I20260811 17:18:14.598407 10975 tablet_replica.cc:330] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f: stopping tablet replica
I20260811 17:18:14.598419 10974 raft_consensus.cc:2229] T 0f65c909ba2c4544ada86e4bf3917419 P 249a0df663e74caf897202149a7da5eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:18:14.598834 10974 raft_consensus.cc:2258] T 0f65c909ba2c4544ada86e4bf3917419 P 249a0df663e74caf897202149a7da5eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:18:14.598919 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 10479
I20260811 17:18:14.598982 10975 raft_consensus.cc:2229] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:18:14.599257 10975 raft_consensus.cc:2258] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:18:14.602429 10975 ts_tablet_manager.cc:1905] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:18:14.604874 10975 ts_tablet_manager.cc:1918] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:18:14.605067 10975 log.cc:1196] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/0f65c909ba2c4544ada86e4bf3917419
I20260811 17:18:14.605528 10975 ts_tablet_manager.cc:1939] T 0f65c909ba2c4544ada86e4bf3917419 P 8298fac5683e46ca9e95192258fcc73f: Deleting consensus metadata
I20260811 17:18:14.606892 10973 tablet_replica.cc:330] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b: stopping tablet replica
I20260811 17:18:14.607044 10394 catalog_manager.cc:4841] TS 8298fac5683e46ca9e95192258fcc73f (127.21.54.131:40873): tablet 0f65c909ba2c4544ada86e4bf3917419 (table test-table [id=7bffd736703247248bc85e002601fc18]) successfully deleted
I20260811 17:18:14.607312 10973 raft_consensus.cc:2229] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:18:14.607673 10973 raft_consensus.cc:2258] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 17:18:14.609063 10396 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.54.129:38235 (error 108)
I20260811 17:18:14.609261 10973 ts_tablet_manager.cc:1905] T 0f65c909ba2c4544ada86e4bf3917419 P 80fc770454ad46b78d16edbc0c470d1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:18:14.609987 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 10633
I20260811 17:18:14.611560 10945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:14.610284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:38235 (local address 127.0.0.1:53474)
0811 17:18:14.611418 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:14.611481 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.129:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1048}
W20260811 17:18:14.611728 10396 catalog_manager.cc:4584] TS 249a0df663e74caf897202149a7da5eb (127.21.54.129:38235): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0f65c909ba2c4544ada86e4bf3917419: Network error: Client connection negotiation failed: client connection to 127.21.54.129:38235: connect: Connection refused (error 111)
I20260811 17:18:14.619534 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 10782
I20260811 17:18:14.621740 10945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:14.619757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:45697 (local address 127.0.0.1:57486)
0811 17:18:14.621539 (+ 1782us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:14.621617 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.130:45697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1649}
I20260811 17:18:14.627811 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 10376
2026-08-11T17:18:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2530 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T17:18:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:14Z Disabled control of system clock
I20260811 17:18:14.665580 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:44571
--webserver_interface=127.21.54.190
--webserver_port=0
--builtin_ntp_servers=127.21.54.148:39235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:44571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 17:18:14.785146 10981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:14.785461 10981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:14.785511 10981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:14.789986 10981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:14.790066 10981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:14.790082 10981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:14.790099 10981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:14.790115 10981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:14.795096 10981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:44571
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:44571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:14.796289 10981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:14.797761 10981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:14.807253 10987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:14.807257 10989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:14.807253 10986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:14.807518 10981 server_base.cc:1034] running on GCE node
I20260811 17:18:14.808313 10981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:14.809659 10981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:14.810849 10981 hybrid_clock.cc:648] HybridClock initialized: now 1786468694810826 us; error 34 us; skew 500 ppm
I20260811 17:18:14.813225 10981 webserver.cc:460] Webserver started at http://127.21.54.190:34037/ using document root <none> and password file <none>
I20260811 17:18:14.813881 10981 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:14.813952 10981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:14.814141 10981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:14.816457 10981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "0749e261af4142438e9bf313fd5b01fe"
format_stamp: "Formatted at 2026-08-11 17:18:14 on dist-test-slave-fd7l"
server_key: "d4d7ea9129d8f99a3ead74a4c1bb07db"
server_key_iv: "78e93fd5d50f2e8d5940c9ec832f3a36"
server_key_version: "encryptionkey@0"
I20260811 17:18:14.817148 10981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "0749e261af4142438e9bf313fd5b01fe"
format_stamp: "Formatted at 2026-08-11 17:18:14 on dist-test-slave-fd7l"
server_key: "d4d7ea9129d8f99a3ead74a4c1bb07db"
server_key_iv: "78e93fd5d50f2e8d5940c9ec832f3a36"
server_key_version: "encryptionkey@0"
I20260811 17:18:14.821378 10981 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:18:14.824265 10999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:14.826120 10981 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 17:18:14.826299 10981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "0749e261af4142438e9bf313fd5b01fe"
format_stamp: "Formatted at 2026-08-11 17:18:14 on dist-test-slave-fd7l"
server_key: "d4d7ea9129d8f99a3ead74a4c1bb07db"
server_key_iv: "78e93fd5d50f2e8d5940c9ec832f3a36"
server_key_version: "encryptionkey@0"
I20260811 17:18:14.826445 10981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:14.853375 10981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:14.854243 10981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:14.854452 10981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:14.863554 10981 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:44571
I20260811 17:18:14.863637 11059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:44571 every 8 connection(s)
I20260811 17:18:14.864851 10981 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:14.868288 11060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:14.873135 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 10981
I20260811 17:18:14.873311 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:14.873704 21722 external_mini_cluster.cc:1428] Setting key fefdc0bb03f2d3b014875e8eeb912df1
I20260811 17:18:14.874469 11060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe: Bootstrap starting.
I20260811 17:18:14.877221 11060 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:14.878170 11060 log.cc:824] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:14.880381 11060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe: No bootstrap required, opened a new log
I20260811 17:18:14.885643 11063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:14.875351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47584 (local address 127.21.54.190:44571)
0811 17:18:14.876120 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:14.876138 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:14.876184 (+ 46us) server_negotiation.cc:407] Connection header received
0811 17:18:14.877035 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:14.877074 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:14.877627 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:14.878047 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:14.878521 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.879856 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.881087 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:14.881774 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:14.881948 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:14.882970 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:14.882980 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:14.883004 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:14.883599 (+ 595us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:14.883692 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:14.883724 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:14.883851 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:14.884021 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 17:18:14.884394 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":133,"threads_started":1}
I20260811 17:18:14.885794 11060 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0749e261af4142438e9bf313fd5b01fe" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44571 } }
I20260811 17:18:14.886116 11060 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:14.886181 11060 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0749e261af4142438e9bf313fd5b01fe, State: Initialized, Role: FOLLOWER
I20260811 17:18:14.886739 11060 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0749e261af4142438e9bf313fd5b01fe" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44571 } }
I20260811 17:18:14.886931 11060 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:14.887015 11060 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:14.887166 11060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:14.888563 11060 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0749e261af4142438e9bf313fd5b01fe" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44571 } }
I20260811 17:18:14.888994 11060 leader_election.cc:302] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0749e261af4142438e9bf313fd5b01fe; no voters:
I20260811 17:18:14.889346 11060 leader_election.cc:288] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:14.889446 11065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:14.889694 11065 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe [term 1 LEADER]: Becoming Leader. State: Replica: 0749e261af4142438e9bf313fd5b01fe, State: Running, Role: LEADER
I20260811 17:18:14.890061 11065 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0749e261af4142438e9bf313fd5b01fe" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44571 } }
I20260811 17:18:14.890830 11060 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:14.891307 11067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0749e261af4142438e9bf313fd5b01fe. Latest consensus state: current_term: 1 leader_uuid: "0749e261af4142438e9bf313fd5b01fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0749e261af4142438e9bf313fd5b01fe" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44571 } } }
I20260811 17:18:14.891466 11067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:14.891687 11066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0749e261af4142438e9bf313fd5b01fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0749e261af4142438e9bf313fd5b01fe" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44571 } } }
I20260811 17:18:14.891803 11066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:14.892040 11069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:14.895469 11069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:14.901887 11069 catalog_manager.cc:1347] Generated new cluster ID: ad22b286959d41738a3e0b88938c5a4a
I20260811 17:18:14.901964 11069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:14.912659 11069 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:14.913731 11069 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:14.919756 11069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe: Generated new TSK 0
I20260811 17:18:14.920663 11069 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:14.931018 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44571
--builtin_ntp_servers=127.21.54.148:39235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 17:18:15.052233 11087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:15.052552 11087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:15.052644 11087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:15.056878 11087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:15.056936 11087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:15.056993 11087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:18:15.061993 11087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:15.063242 11087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:15.064802 11087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:15.075325 11093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:15.075397 11092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:15.075393 11087 server_base.cc:1034] running on GCE node
W20260811 17:18:15.075424 11095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:15.076076 11087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:15.076870 11087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:15.078081 11087 hybrid_clock.cc:648] HybridClock initialized: now 1786468695078060 us; error 31 us; skew 500 ppm
I20260811 17:18:15.080291 11087 webserver.cc:460] Webserver started at http://127.21.54.129:41437/ using document root <none> and password file <none>
I20260811 17:18:15.080986 11087 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:15.081089 11087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:15.081339 11087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:15.083568 11087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "52b3d7c82f7f41baad26e4b0f7c2303b"
format_stamp: "Formatted at 2026-08-11 17:18:15 on dist-test-slave-fd7l"
server_key: "5d33c30cb7ba37b7e51999f8a6da79c0"
server_key_iv: "e5e9ff562f505e9bbd7805f6428f3eed"
server_key_version: "encryptionkey@0"
I20260811 17:18:15.084164 11087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "52b3d7c82f7f41baad26e4b0f7c2303b"
format_stamp: "Formatted at 2026-08-11 17:18:15 on dist-test-slave-fd7l"
server_key: "5d33c30cb7ba37b7e51999f8a6da79c0"
server_key_iv: "e5e9ff562f505e9bbd7805f6428f3eed"
server_key_version: "encryptionkey@0"
I20260811 17:18:15.088189 11087 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 17:18:15.091219 11101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:15.092861 11087 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:15.093043 11087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "52b3d7c82f7f41baad26e4b0f7c2303b"
format_stamp: "Formatted at 2026-08-11 17:18:15 on dist-test-slave-fd7l"
server_key: "5d33c30cb7ba37b7e51999f8a6da79c0"
server_key_iv: "e5e9ff562f505e9bbd7805f6428f3eed"
server_key_version: "encryptionkey@0"
I20260811 17:18:15.093181 11087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:15.106282 11087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:15.107060 11087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:15.107291 11087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:15.107957 11087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:18:15.109176 11087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:18:15.109256 11087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:15.109340 11087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:18:15.109387 11087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:15.119717 11087 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:35381
I20260811 17:18:15.119740 11227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:35381 every 8 connection(s)
I20260811 17:18:15.120922 11087 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:18:15.129498 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 11087
I20260811 17:18:15.129671 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:18:15.130079 21722 external_mini_cluster.cc:1428] Setting key 7719e9269d901d9dcf33b3d28cf053ea
I20260811 17:18:15.132215 11063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:15.123213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:36017 (local address 127.21.54.190:44571)
0811 17:18:15.123384 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:15.123389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:15.124606 (+ 1217us) server_negotiation.cc:407] Connection header received
0811 17:18:15.126163 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:15.126167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:15.126221 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:15.126312 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:15.127493 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.128103 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.129411 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.129913 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.130057 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:15.131553 (+ 1496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:15.131559 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:15.131562 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:15.131584 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:15.131606 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:15.131656 (+ 50us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:15.131745 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:15.131980 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 17:18:15.132050 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 17:18:15.132673 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44571
--builtin_ntp_servers=127.21.54.148:39235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 17:18:15.133373 11230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:15.123707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:44571 (local address 127.21.54.129:36017)
0811 17:18:15.124389 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:15.124432 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:15.125920 (+ 1488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:15.126648 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:15.126656 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:15.126749 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:15.127305 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:15.127320 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.128237 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.128240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:15.129256 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:15.129267 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.130091 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.130095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:15.130310 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:15.131197 (+ 887us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:18:15.131199 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:15.131245 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:18:15.131320 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:15.131768 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:15.131780 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:15.131787 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 17:18:15.131939 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 17:18:15.132224 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":122,"threads_started":1}
I20260811 17:18:15.134840 11228 heartbeater.cc:347] Connected to a master server at 127.21.54.190:44571
I20260811 17:18:15.135133 11228 heartbeater.cc:464] Registering TS with master...
I20260811 17:18:15.135751 11228 heartbeater.cc:511] Master 127.21.54.190:44571 requested a full tablet report, sending...
I20260811 17:18:15.137643 11018 ts_manager.cc:194] Registered new tserver with Master: 52b3d7c82f7f41baad26e4b0f7c2303b (127.21.54.129:35381)
I20260811 17:18:15.139675 11018 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.54.129:36017
W20260811 17:18:15.255586 11231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:15.255882 11231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:15.255921 11231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:15.260203 11231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:15.260285 11231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:15.260345 11231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:18:15.265479 11231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:15.266711 11231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:15.268337 11231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:15.278807 11241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:15.278825 11238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:15.278851 11231 server_base.cc:1034] running on GCE node
W20260811 17:18:15.278807 11239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:15.279637 11231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:15.280336 11231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:15.281545 11231 hybrid_clock.cc:648] HybridClock initialized: now 1786468695281527 us; error 35 us; skew 500 ppm
I20260811 17:18:15.283735 11231 webserver.cc:460] Webserver started at http://127.21.54.130:42647/ using document root <none> and password file <none>
I20260811 17:18:15.284372 11231 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:15.284473 11231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:15.284756 11231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:15.286960 11231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "61e4060c540443cd913fea1461b490b8"
format_stamp: "Formatted at 2026-08-11 17:18:15 on dist-test-slave-fd7l"
server_key: "7c0a89383c659820ef2b71cf00dbf669"
server_key_iv: "68e3a7c44a12c885fe99721cb1be84de"
server_key_version: "encryptionkey@0"
I20260811 17:18:15.287581 11231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "61e4060c540443cd913fea1461b490b8"
format_stamp: "Formatted at 2026-08-11 17:18:15 on dist-test-slave-fd7l"
server_key: "7c0a89383c659820ef2b71cf00dbf669"
server_key_iv: "68e3a7c44a12c885fe99721cb1be84de"
server_key_version: "encryptionkey@0"
I20260811 17:18:15.292114 11231 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 17:18:15.295203 11247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:15.296660 11231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:15.296798 11231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "61e4060c540443cd913fea1461b490b8"
format_stamp: "Formatted at 2026-08-11 17:18:15 on dist-test-slave-fd7l"
server_key: "7c0a89383c659820ef2b71cf00dbf669"
server_key_iv: "68e3a7c44a12c885fe99721cb1be84de"
server_key_version: "encryptionkey@0"
I20260811 17:18:15.296924 11231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:15.315986 11231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:15.316794 11231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:15.317016 11231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:15.317667 11231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:18:15.318670 11231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:18:15.318744 11231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:15.318814 11231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:18:15.318872 11231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:15.328354 11231 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:39381
I20260811 17:18:15.328385 11376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:39381 every 8 connection(s)
I20260811 17:18:15.329351 11231 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:18:15.332113 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 11231
I20260811 17:18:15.332295 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:18:15.332742 21722 external_mini_cluster.cc:1428] Setting key 5620a312164fb20ac5015be52af1dc43
I20260811 17:18:15.335003 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44571
--builtin_ntp_servers=127.21.54.148:39235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 17:18:15.340003 11063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:15.331457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:42787 (local address 127.21.54.190:44571)
0811 17:18:15.331624 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:15.331630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:15.332841 (+ 1211us) server_negotiation.cc:407] Connection header received
0811 17:18:15.334298 (+ 1457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:15.334301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:15.334358 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:15.334466 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:15.335476 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.336009 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.337717 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.337983 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.338091 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:15.339365 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:15.339371 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:15.339377 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:15.339396 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:15.339424 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:15.339438 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:15.339521 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:15.339699 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 17:18:15.339777 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 17:18:15.340905 11379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:15.331815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:44571 (local address 127.21.54.130:42787)
0811 17:18:15.332656 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:15.332711 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:15.334071 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:15.334655 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:15.334663 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:15.334748 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:15.335317 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:15.335334 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.336155 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.336159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:15.337540 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:15.337567 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.338158 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.338165 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:15.338329 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:15.339064 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:18:15.339066 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:15.339108 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:18:15.339177 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:15.339536 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:15.339544 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:15.339549 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 17:18:15.339660 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 17:18:15.339901 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":143,"threads_started":1}
I20260811 17:18:15.342278 11377 heartbeater.cc:347] Connected to a master server at 127.21.54.190:44571
I20260811 17:18:15.342545 11377 heartbeater.cc:464] Registering TS with master...
I20260811 17:18:15.343154 11377 heartbeater.cc:511] Master 127.21.54.190:44571 requested a full tablet report, sending...
I20260811 17:18:15.344446 11018 ts_manager.cc:194] Registered new tserver with Master: 61e4060c540443cd913fea1461b490b8 (127.21.54.130:39381)
I20260811 17:18:15.345152 11018 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.54.130:42787
W20260811 17:18:15.455905 11380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:15.456182 11380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:15.456223 11380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:15.460285 11380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:15.460352 11380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:15.460409 11380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:18:15.468022 11380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:44571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:15.469494 11380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:15.471724 11380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:15.483176 11391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:15.483285 11387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:15.483287 11389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:15.483680 11380 server_base.cc:1034] running on GCE node
I20260811 17:18:15.484176 11380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:15.484968 11380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:15.486159 11380 hybrid_clock.cc:648] HybridClock initialized: now 1786468695486122 us; error 49 us; skew 500 ppm
I20260811 17:18:15.488152 11380 webserver.cc:460] Webserver started at http://127.21.54.131:38541/ using document root <none> and password file <none>
I20260811 17:18:15.488849 11380 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:15.488948 11380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:15.489204 11380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:15.491504 11380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "bd814eeac2324ec084180bfa177bfa6b"
format_stamp: "Formatted at 2026-08-11 17:18:15 on dist-test-slave-fd7l"
server_key: "e6357450583fd3cb7639f2bcc1ffea93"
server_key_iv: "00c54d148a2850928f64d0f90e7014c4"
server_key_version: "encryptionkey@0"
I20260811 17:18:15.492134 11380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "bd814eeac2324ec084180bfa177bfa6b"
format_stamp: "Formatted at 2026-08-11 17:18:15 on dist-test-slave-fd7l"
server_key: "e6357450583fd3cb7639f2bcc1ffea93"
server_key_iv: "00c54d148a2850928f64d0f90e7014c4"
server_key_version: "encryptionkey@0"
I20260811 17:18:15.496143 11380 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 17:18:15.498994 11397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:15.500365 11380 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:18:15.500577 11380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "bd814eeac2324ec084180bfa177bfa6b"
format_stamp: "Formatted at 2026-08-11 17:18:15 on dist-test-slave-fd7l"
server_key: "e6357450583fd3cb7639f2bcc1ffea93"
server_key_iv: "00c54d148a2850928f64d0f90e7014c4"
server_key_version: "encryptionkey@0"
I20260811 17:18:15.500712 11380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:15.519271 11380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:15.520071 11380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:15.520290 11380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:15.521023 11380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:18:15.522073 11380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:18:15.522166 11380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:15.522264 11380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:18:15.522315 11380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:15.531569 11380 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:42525
I20260811 17:18:15.531617 11532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:42525 every 8 connection(s)
I20260811 17:18:15.532661 11380 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:18:15.541544 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 11380
I20260811 17:18:15.541659 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:18:15.542053 21722 external_mini_cluster.cc:1428] Setting key cc1f5e7a7215f9e15c13d896ebd5c0b9
I20260811 17:18:15.542831 11063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:15.534792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:50751 (local address 127.21.54.190:44571)
0811 17:18:15.534944 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:15.534948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:15.536057 (+ 1109us) server_negotiation.cc:407] Connection header received
0811 17:18:15.537601 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:15.537604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:15.537670 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:15.537780 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:15.538759 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.539295 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.540548 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.540784 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.540907 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:15.542275 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:15.542278 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:15.542281 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:15.542295 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:15.542316 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:15.542324 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:15.542387 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:15.542610 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 17:18:15.542687 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 17:18:15.543874 11535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:15.535118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:44571 (local address 127.21.54.131:50751)
0811 17:18:15.535864 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:15.535908 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:15.537346 (+ 1438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:15.537977 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:15.537984 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:15.538081 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:15.538597 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:15.538610 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.539436 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.539440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:15.540377 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:15.540387 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.540955 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.540959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:15.541144 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:15.541896 (+ 752us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:18:15.541899 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:15.541961 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:18:15.542050 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:15.542430 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:15.542443 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:15.542449 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 17:18:15.542573 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 17:18:15.542885 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":124,"threads_started":1}
I20260811 17:18:15.545138 11533 heartbeater.cc:347] Connected to a master server at 127.21.54.190:44571
I20260811 17:18:15.545413 11533 heartbeater.cc:464] Registering TS with master...
I20260811 17:18:15.546003 11533 heartbeater.cc:511] Master 127.21.54.190:44571 requested a full tablet report, sending...
I20260811 17:18:15.547216 11018 ts_manager.cc:194] Registered new tserver with Master: bd814eeac2324ec084180bfa177bfa6b (127.21.54.131:42525)
I20260811 17:18:15.547819 11018 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.54.131:50751
I20260811 17:18:15.556587 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:18:15.563213 11063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:15.558713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47586 (local address 127.21.54.190:44571)
0811 17:18:15.558857 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:15.558861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:15.559120 (+ 259us) server_negotiation.cc:407] Connection header received
0811 17:18:15.559194 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:15.559196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:15.559240 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:15.559313 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:15.559626 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.560145 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.561227 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.561444 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.561525 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:15.562666 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:15.562669 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:15.562671 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:15.562685 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:15.562704 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:15.562712 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:15.562818 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:15.563024 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 17:18:15.563066 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 17:18:15.566558 11018 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:18:15.569636 11018 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:18:15.585322 11554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:15.579517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49866 (local address 127.21.54.129:35381)
0811 17:18:15.580094 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:15.580105 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:15.580355 (+ 250us) server_negotiation.cc:407] Connection header received
0811 17:18:15.580606 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:15.580612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:15.580834 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:15.580957 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:15.581327 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.582038 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.583580 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.584355 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.584465 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:15.584715 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:15.585007 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 17:18:15.585134 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":128,"threads_started":1}
I20260811 17:18:15.585608 11550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:15.578701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:42525 (local address 127.0.0.1:52890)
0811 17:18:15.579589 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:15.579644 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:15.579784 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:15.580420 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:15.580427 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:15.580449 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:15.580636 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:15.580644 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.582161 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.582164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:15.583897 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:15.583909 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.585117 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.585120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:15.585210 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:15.585233 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 17:18:15.585297 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 17:18:15.585430 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":202,"thread_start_us":87,"threads_started":1}
I20260811 17:18:15.585753 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:15.579106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:35381 (local address 127.0.0.1:49866)
0811 17:18:15.580263 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:15.580277 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:15.580423 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:15.581141 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:15.581144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:15.581158 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:15.581219 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:15.581223 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.582210 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.582213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:15.583440 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:15.583450 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.584490 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.584494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:15.584681 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:15.584758 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 17:18:15.585543 (+ 785us) client_negotiation.cc:240] Negotiation successful
0811 17:18:15.585625 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1094,"mutex_wait_us":75,"thread_start_us":91,"threads_started":1}
I20260811 17:18:15.586120 11553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:15.579395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41738 (local address 127.21.54.130:39381)
0811 17:18:15.580340 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:15.580357 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:15.580382 (+ 25us) server_negotiation.cc:407] Connection header received
0811 17:18:15.580455 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:15.580460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:15.580684 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:15.580865 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:15.581108 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.582194 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.584066 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.584996 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.585136 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:15.585320 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:15.585824 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 17:18:15.585929 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":833,"thread_start_us":172,"threads_started":1}
I20260811 17:18:15.586370 11552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:15.578886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:39381 (local address 127.0.0.1:41738)
0811 17:18:15.579982 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:15.579996 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:15.580228 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:15.580855 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:15.580857 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:15.580876 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:15.580979 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:15.580985 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.582325 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.582329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:15.583900 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:15.583909 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.585380 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.585383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:15.585549 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:15.585573 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 17:18:15.585648 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 17:18:15.585720 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1026,"mutex_wait_us":188,"thread_start_us":111,"threads_started":1}
I20260811 17:18:15.586382 11549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:15.578866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52890 (local address 127.21.54.131:42525)
0811 17:18:15.579920 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:15.579935 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:15.579963 (+ 28us) server_negotiation.cc:407] Connection header received
0811 17:18:15.580044 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:15.580050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:15.580267 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:15.580434 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:15.580766 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.581992 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.584058 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.584986 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.585090 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:15.585263 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:15.585320 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 17:18:15.586231 (+ 911us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":940,"thread_start_us":126,"threads_started":1}
I20260811 17:18:15.589112 11155 tablet_service.cc:1467] Processing CreateTablet for tablet 5b3df04ec982444e9063f5d897fc34e4 (DEFAULT_TABLE table=test-table [id=9dc75829f833446db45bda9797cea5f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:18:15.589140 11304 tablet_service.cc:1467] Processing CreateTablet for tablet 5b3df04ec982444e9063f5d897fc34e4 (DEFAULT_TABLE table=test-table [id=9dc75829f833446db45bda9797cea5f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:18:15.590191 11155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b3df04ec982444e9063f5d897fc34e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:15.590219 11304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b3df04ec982444e9063f5d897fc34e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:15.590602 11454 tablet_service.cc:1467] Processing CreateTablet for tablet 5b3df04ec982444e9063f5d897fc34e4 (DEFAULT_TABLE table=test-table [id=9dc75829f833446db45bda9797cea5f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:18:15.591601 11454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b3df04ec982444e9063f5d897fc34e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:15.597255 11556 tablet_bootstrap.cc:492] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b: Bootstrap starting.
I20260811 17:18:15.598842 11557 tablet_bootstrap.cc:492] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8: Bootstrap starting.
I20260811 17:18:15.599603 11556 tablet_bootstrap.cc:654] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:15.600466 11556 log.cc:824] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:15.600698 11557 tablet_bootstrap.cc:654] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:15.600915 11558 tablet_bootstrap.cc:492] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b: Bootstrap starting.
I20260811 17:18:15.601498 11557 log.cc:824] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:15.602948 11556 tablet_bootstrap.cc:492] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b: No bootstrap required, opened a new log
I20260811 17:18:15.603075 11558 tablet_bootstrap.cc:654] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:15.603161 11556 ts_tablet_manager.cc:1397] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 17:18:15.603652 11557 tablet_bootstrap.cc:492] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8: No bootstrap required, opened a new log
I20260811 17:18:15.603849 11557 ts_tablet_manager.cc:1397] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260811 17:18:15.603856 11558 log.cc:824] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:15.605638 11558 tablet_bootstrap.cc:492] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b: No bootstrap required, opened a new log
I20260811 17:18:15.605832 11558 ts_tablet_manager.cc:1397] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 17:18:15.607843 11556 raft_consensus.cc:348] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd814eeac2324ec084180bfa177bfa6b" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 42525 } } peers { permanent_uuid: "61e4060c540443cd913fea1461b490b8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39381 } } peers { permanent_uuid: "52b3d7c82f7f41baad26e4b0f7c2303b" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35381 } }
I20260811 17:18:15.608139 11556 raft_consensus.cc:374] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:15.608196 11556 raft_consensus.cc:729] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b3d7c82f7f41baad26e4b0f7c2303b, State: Initialized, Role: FOLLOWER
I20260811 17:18:15.608078 11557 raft_consensus.cc:348] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd814eeac2324ec084180bfa177bfa6b" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 42525 } } peers { permanent_uuid: "61e4060c540443cd913fea1461b490b8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39381 } } peers { permanent_uuid: "52b3d7c82f7f41baad26e4b0f7c2303b" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35381 } }
I20260811 17:18:15.608610 11557 raft_consensus.cc:374] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:15.608664 11557 raft_consensus.cc:729] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61e4060c540443cd913fea1461b490b8, State: Initialized, Role: FOLLOWER
I20260811 17:18:15.608752 11556 consensus_queue.cc:260] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd814eeac2324ec084180bfa177bfa6b" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 42525 } } peers { permanent_uuid: "61e4060c540443cd913fea1461b490b8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39381 } } peers { permanent_uuid: "52b3d7c82f7f41baad26e4b0f7c2303b" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35381 } }
I20260811 17:18:15.609361 11557 consensus_queue.cc:260] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd814eeac2324ec084180bfa177bfa6b" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 42525 } } peers { permanent_uuid: "61e4060c540443cd913fea1461b490b8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39381 } } peers { permanent_uuid: "52b3d7c82f7f41baad26e4b0f7c2303b" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35381 } }
I20260811 17:18:15.609449 11558 raft_consensus.cc:348] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd814eeac2324ec084180bfa177bfa6b" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 42525 } } peers { permanent_uuid: "61e4060c540443cd913fea1461b490b8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39381 } } peers { permanent_uuid: "52b3d7c82f7f41baad26e4b0f7c2303b" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35381 } }
I20260811 17:18:15.609720 11558 raft_consensus.cc:374] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:15.609781 11558 raft_consensus.cc:729] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd814eeac2324ec084180bfa177bfa6b, State: Initialized, Role: FOLLOWER
I20260811 17:18:15.609797 11556 ts_tablet_manager.cc:1428] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 17:18:15.610177 11228 heartbeater.cc:503] Master 127.21.54.190:44571 was elected leader, sending a full tablet report...
I20260811 17:18:15.610283 11557 ts_tablet_manager.cc:1428] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 17:18:15.610203 11558 consensus_queue.cc:260] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd814eeac2324ec084180bfa177bfa6b" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 42525 } } peers { permanent_uuid: "61e4060c540443cd913fea1461b490b8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39381 } } peers { permanent_uuid: "52b3d7c82f7f41baad26e4b0f7c2303b" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35381 } }
I20260811 17:18:15.610605 11377 heartbeater.cc:503] Master 127.21.54.190:44571 was elected leader, sending a full tablet report...
I20260811 17:18:15.611135 11558 ts_tablet_manager.cc:1428] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 17:18:15.611424 11533 heartbeater.cc:503] Master 127.21.54.190:44571 was elected leader, sending a full tablet report...
W20260811 17:18:15.622270 11229 tablet.cc:2369] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:18:15.783828 11534 tablet.cc:2369] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:18:15.830621 11378 tablet.cc:2369] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:18:15.830870 11563 raft_consensus.cc:482] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:18:15.831131 11563 raft_consensus.cc:504] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd814eeac2324ec084180bfa177bfa6b" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 42525 } } peers { permanent_uuid: "61e4060c540443cd913fea1461b490b8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39381 } } peers { permanent_uuid: "52b3d7c82f7f41baad26e4b0f7c2303b" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35381 } }
I20260811 17:18:15.832440 11563 leader_election.cc:288] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd814eeac2324ec084180bfa177bfa6b (127.21.54.131:42525), 61e4060c540443cd913fea1461b490b8 (127.21.54.130:39381)
I20260811 17:18:15.836589 11549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:15.832801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:55589 (local address 127.21.54.131:42525)
0811 17:18:15.832964 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:15.832969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:15.833213 (+ 244us) server_negotiation.cc:407] Connection header received
0811 17:18:15.833340 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:15.833343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:15.833396 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:15.833482 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:15.833778 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.834348 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.835506 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.835946 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.836049 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:15.836131 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:15.836321 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 17:18:15.836411 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 17:18:15.836604 11567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:15.832785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:39381 (local address 127.21.54.129:42681)
0811 17:18:15.833200 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:15.833217 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:15.833334 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:15.833698 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:15.833700 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:15.833715 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:15.833857 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:15.833863 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.834647 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.834650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:15.835448 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:15.835461 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.836155 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.836159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:15.836253 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:15.836299 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 17:18:15.836379 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 17:18:15.836440 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":128,"threads_started":1}
I20260811 17:18:15.836630 11553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:15.832891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:42681 (local address 127.21.54.130:39381)
0811 17:18:15.833035 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:15.833040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:15.833327 (+ 287us) server_negotiation.cc:407] Connection header received
0811 17:18:15.833463 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:15.833468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:15.833581 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:15.833668 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:15.833975 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.834501 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.835599 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.835983 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.836125 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:15.836177 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:15.836418 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 17:18:15.836491 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 17:18:15.837069 11566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:15.832692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:42525 (local address 127.21.54.129:55589)
0811 17:18:15.833097 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:15.833117 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:15.833227 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:15.833496 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:15.833499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:15.833515 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:15.833665 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:15.833672 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.834455 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.834458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:15.835362 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:15.835372 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.836085 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.836089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:15.836180 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:15.836210 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 17:18:15.836284 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 17:18:15.836343 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":95,"threads_started":1}
I20260811 17:18:15.837478 11325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3df04ec982444e9063f5d897fc34e4" candidate_uuid: "52b3d7c82f7f41baad26e4b0f7c2303b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61e4060c540443cd913fea1461b490b8" is_pre_election: true
I20260811 17:18:15.837699 11477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3df04ec982444e9063f5d897fc34e4" candidate_uuid: "52b3d7c82f7f41baad26e4b0f7c2303b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd814eeac2324ec084180bfa177bfa6b" is_pre_election: true
I20260811 17:18:15.837852 11325 raft_consensus.cc:2454] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b3d7c82f7f41baad26e4b0f7c2303b in term 0.
I20260811 17:18:15.837998 11477 raft_consensus.cc:2454] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b3d7c82f7f41baad26e4b0f7c2303b in term 0.
I20260811 17:18:15.838420 11102 leader_election.cc:302] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 52b3d7c82f7f41baad26e4b0f7c2303b, 61e4060c540443cd913fea1461b490b8, bd814eeac2324ec084180bfa177bfa6b; no voters:
I20260811 17:18:15.838725 11563 raft_consensus.cc:2781] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:18:15.838806 11563 raft_consensus.cc:482] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:18:15.838836 11563 raft_consensus.cc:3037] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:15.839959 11563 raft_consensus.cc:504] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd814eeac2324ec084180bfa177bfa6b" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 42525 } } peers { permanent_uuid: "61e4060c540443cd913fea1461b490b8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39381 } } peers { permanent_uuid: "52b3d7c82f7f41baad26e4b0f7c2303b" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35381 } }
I20260811 17:18:15.840354 11563 leader_election.cc:288] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [CANDIDATE]: Term 1 election: Requested vote from peers bd814eeac2324ec084180bfa177bfa6b (127.21.54.131:42525), 61e4060c540443cd913fea1461b490b8 (127.21.54.130:39381)
I20260811 17:18:15.840801 11477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3df04ec982444e9063f5d897fc34e4" candidate_uuid: "52b3d7c82f7f41baad26e4b0f7c2303b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd814eeac2324ec084180bfa177bfa6b"
I20260811 17:18:15.840922 11477 raft_consensus.cc:3037] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:15.841929 11477 raft_consensus.cc:2454] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b3d7c82f7f41baad26e4b0f7c2303b in term 1.
I20260811 17:18:15.842228 11325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3df04ec982444e9063f5d897fc34e4" candidate_uuid: "52b3d7c82f7f41baad26e4b0f7c2303b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61e4060c540443cd913fea1461b490b8"
I20260811 17:18:15.842303 11105 leader_election.cc:302] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b3d7c82f7f41baad26e4b0f7c2303b, bd814eeac2324ec084180bfa177bfa6b; no voters:
I20260811 17:18:15.842365 11325 raft_consensus.cc:3037] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:15.842491 11563 raft_consensus.cc:2781] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:15.842742 11563 raft_consensus.cc:686] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [term 1 LEADER]: Becoming Leader. State: Replica: 52b3d7c82f7f41baad26e4b0f7c2303b, State: Running, Role: LEADER
I20260811 17:18:15.843135 11563 consensus_queue.cc:237] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd814eeac2324ec084180bfa177bfa6b" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 42525 } } peers { permanent_uuid: "61e4060c540443cd913fea1461b490b8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39381 } } peers { permanent_uuid: "52b3d7c82f7f41baad26e4b0f7c2303b" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35381 } }
I20260811 17:18:15.843534 11325 raft_consensus.cc:2454] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b3d7c82f7f41baad26e4b0f7c2303b in term 1.
I20260811 17:18:15.846621 11018 catalog_manager.cc:5495] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b reported cstate change: term changed from 0 to 1, leader changed from <none> to 52b3d7c82f7f41baad26e4b0f7c2303b (127.21.54.129). New cstate: current_term: 1 leader_uuid: "52b3d7c82f7f41baad26e4b0f7c2303b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd814eeac2324ec084180bfa177bfa6b" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 42525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61e4060c540443cd913fea1461b490b8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b3d7c82f7f41baad26e4b0f7c2303b" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35381 } health_report { overall_health: HEALTHY } } }
I20260811 17:18:15.881596 11554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:15.877649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49870 (local address 127.21.54.129:35381)
0811 17:18:15.877793 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:15.877797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:15.877811 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:18:15.877965 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:15.877967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:15.878026 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:15.878133 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:18:15.878347 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.878876 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.879927 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:15.880148 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:15.880234 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:15.880636 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:18:15.881080 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:18:15.881248 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:15.881405 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 17:18:15.881443 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 17:18:15.888070 11325 raft_consensus.cc:1261] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8 [term 1 FOLLOWER]: Refusing update from remote peer 52b3d7c82f7f41baad26e4b0f7c2303b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:18:15.888070 11477 raft_consensus.cc:1261] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b [term 1 FOLLOWER]: Refusing update from remote peer 52b3d7c82f7f41baad26e4b0f7c2303b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:18:15.888959 11568 consensus_queue.cc:1035] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [LEADER]: Connected to new peer: Peer: permanent_uuid: "61e4060c540443cd913fea1461b490b8" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:18:15.889261 11563 consensus_queue.cc:1035] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd814eeac2324ec084180bfa177bfa6b" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 42525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:18:15.899436 11572 mvcc.cc:205] Tried to move back new op lower bound from 7317375778349723648 to 7317375778176425984. Current Snapshot: MvccSnapshot[applied={T|T < 7317375778349723648}]
I20260811 17:18:15.900962 11573 mvcc.cc:205] Tried to move back new op lower bound from 7317375778349723648 to 7317375778176425984. Current Snapshot: MvccSnapshot[applied={T|T < 7317375778349723648}]
I20260811 17:18:15.908270 11018 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:18:15.908485 11018 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:18:15.911402 11018 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0749e261af4142438e9bf313fd5b01fe: Sending DeleteTablet for 3 replicas of tablet 5b3df04ec982444e9063f5d897fc34e4
I20260811 17:18:15.912355 11304 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b3df04ec982444e9063f5d897fc34e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:18:15 UTC) from {username='slave'} at 127.0.0.1:41738
I20260811 17:18:15.912367 11454 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b3df04ec982444e9063f5d897fc34e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:18:15 UTC) from {username='slave'} at 127.0.0.1:52890
I20260811 17:18:15.912937 11582 tablet_replica.cc:330] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8: stopping tablet replica
I20260811 17:18:15.913286 11582 raft_consensus.cc:2229] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:18:15.913478 11155 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b3df04ec982444e9063f5d897fc34e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:18:15 UTC) from {username='slave'} at 127.0.0.1:49866
I20260811 17:18:15.913688 11582 raft_consensus.cc:2258] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:18:15.914068 11584 tablet_replica.cc:330] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b: stopping tablet replica
I20260811 17:18:15.914309 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 11087
I20260811 17:18:15.914502 11584 raft_consensus.cc:2229] T 5b3df04ec982444e9063f5d897fc34e4 P 52b3d7c82f7f41baad26e4b0f7c2303b [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:18:15.915413 11582 ts_tablet_manager.cc:1905] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:18:15.918455 11582 ts_tablet_manager.cc:1918] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:18:15.918599 11582 log.cc:1196] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/5b3df04ec982444e9063f5d897fc34e4
I20260811 17:18:15.918926 11582 ts_tablet_manager.cc:1939] T 5b3df04ec982444e9063f5d897fc34e4 P 61e4060c540443cd913fea1461b490b8: Deleting consensus metadata
I20260811 17:18:15.920159 11000 catalog_manager.cc:4841] TS 61e4060c540443cd913fea1461b490b8 (127.21.54.130:39381): tablet 5b3df04ec982444e9063f5d897fc34e4 (table test-table [id=9dc75829f833446db45bda9797cea5f2]) successfully deleted
I20260811 17:18:15.920784 11583 tablet_replica.cc:330] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b: stopping tablet replica
I20260811 17:18:15.921149 11583 raft_consensus.cc:2229] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:18:15.921482 11583 raft_consensus.cc:2258] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:18:15.922950 11583 ts_tablet_manager.cc:1905] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:18:15.925251 11583 ts_tablet_manager.cc:1918] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:18:15.925333 11583 log.cc:1196] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/5b3df04ec982444e9063f5d897fc34e4
I20260811 17:18:15.925626 11583 ts_tablet_manager.cc:1939] T 5b3df04ec982444e9063f5d897fc34e4 P bd814eeac2324ec084180bfa177bfa6b: Deleting consensus metadata
W20260811 17:18:15.926242 11001 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.54.129:35381 (error 108)
I20260811 17:18:15.927013 11002 catalog_manager.cc:4841] TS bd814eeac2324ec084180bfa177bfa6b (127.21.54.131:42525): tablet 5b3df04ec982444e9063f5d897fc34e4 (table test-table [id=9dc75829f833446db45bda9797cea5f2]) successfully deleted
I20260811 17:18:15.927388 11552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:15.927072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:35381 (local address 127.0.0.1:49880)
0811 17:18:15.927242 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:15.927313 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.129:35381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 17:18:15.927525 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 11231
W20260811 17:18:15.927609 11001 catalog_manager.cc:4584] TS 52b3d7c82f7f41baad26e4b0f7c2303b (127.21.54.129:35381): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b3df04ec982444e9063f5d897fc34e4: Network error: Client connection negotiation failed: client connection to 127.21.54.129:35381: connect: Connection refused (error 111)
I20260811 17:18:15.936914 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 11380
I20260811 17:18:15.946574 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 10981
2026-08-11T17:18:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1317 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:16 dist-test-slave-fd7l krb5kdc[11590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:16 dist-test-slave-fd7l krb5kdc[11590](info): set up 2 sockets
Aug 11 17:18:16 dist-test-slave-fd7l krb5kdc[11590](info): commencing operation
krb5kdc: starting...
W20260811 17:18:17.164897 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:17 dist-test-slave-fd7l krb5kdc[11590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:18:17.304459 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:38013
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:33525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:38013
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:17.426930 11606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:17.427241 11606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:17.427284 11606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:17.431649 11606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:17.431756 11606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:17.431777 11606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:17.431794 11606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:17.431809 11606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:17.437572 11606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:33525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:38013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:38013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:17.438777 11606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:17.439874 11606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:18:17.449375 11606 server_base.cc:1034] running on GCE node
W20260811 17:18:17.449271 11614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:17.449271 11613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:17.449256 11616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:17.450232 11606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:17.451535 11606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:17.452752 11606 hybrid_clock.cc:648] HybridClock initialized: now 1786468697452732 us; error 32 us; skew 500 ppm
Aug 11 17:18:17 dist-test-slave-fd7l krb5kdc[11590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468697, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:17.456158 11606 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:18:17.457209 11606 webserver.cc:460] Webserver started at http://127.21.54.190:38323/ using document root <none> and password file <none>
I20260811 17:18:17.457829 11606 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:17.457922 11606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:17.458168 11606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:17.460335 11606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "00a9a586c8254e499dfc2a3464727dba"
format_stamp: "Formatted at 2026-08-11 17:18:17 on dist-test-slave-fd7l"
server_key: "d3faa8cd1049e3089517bd5ae1e1b280"
server_key_iv: "93766361778c0fc04e249ee42c0a7ab6"
server_key_version: "encryptionkey@0"
I20260811 17:18:17.461037 11606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "00a9a586c8254e499dfc2a3464727dba"
format_stamp: "Formatted at 2026-08-11 17:18:17 on dist-test-slave-fd7l"
server_key: "d3faa8cd1049e3089517bd5ae1e1b280"
server_key_iv: "93766361778c0fc04e249ee42c0a7ab6"
server_key_version: "encryptionkey@0"
I20260811 17:18:17.467836 11606 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260811 17:18:17.470911 11623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:17.472443 11606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:18:17.472674 11606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "00a9a586c8254e499dfc2a3464727dba"
format_stamp: "Formatted at 2026-08-11 17:18:17 on dist-test-slave-fd7l"
server_key: "d3faa8cd1049e3089517bd5ae1e1b280"
server_key_iv: "93766361778c0fc04e249ee42c0a7ab6"
server_key_version: "encryptionkey@0"
I20260811 17:18:17.472815 11606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:17.501360 11606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:17.504935 11606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:17.505162 11606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:17.513720 11606 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:38013
I20260811 17:18:17.513748 11685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:38013 every 8 connection(s)
I20260811 17:18:17.514865 11606 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:17.517890 11686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:17.523356 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 11606
I20260811 17:18:17.523509 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:17.523808 21722 external_mini_cluster.cc:1428] Setting key f9d082e73a63c922bf3d9770cbcb98aa
I20260811 17:18:17.524080 11686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba: Bootstrap starting.
I20260811 17:18:17.526932 11686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:17.527848 11686 log.cc:824] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:17.530622 11686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba: No bootstrap required, opened a new log
Aug 11 17:18:17 dist-test-slave-fd7l krb5kdc[11590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:18:17.534776 11686 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a9a586c8254e499dfc2a3464727dba" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38013 } }
I20260811 17:18:17.535046 11686 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:17.535104 11686 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00a9a586c8254e499dfc2a3464727dba, State: Initialized, Role: FOLLOWER
I20260811 17:18:17.535627 11686 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a9a586c8254e499dfc2a3464727dba" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38013 } }
I20260811 17:18:17.535826 11686 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:17.535910 11686 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:17.536085 11686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:17.537178 11686 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a9a586c8254e499dfc2a3464727dba" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38013 } }
I20260811 17:18:17.537551 11686 leader_election.cc:302] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00a9a586c8254e499dfc2a3464727dba; no voters:
I20260811 17:18:17.537925 11686 leader_election.cc:288] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:17.538064 11691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:17.538269 11691 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba [term 1 LEADER]: Becoming Leader. State: Replica: 00a9a586c8254e499dfc2a3464727dba, State: Running, Role: LEADER
I20260811 17:18:17.538587 11691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a9a586c8254e499dfc2a3464727dba" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38013 } }
I20260811 17:18:17.539347 11686 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:17.541834 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:17.525403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34670 (local address 127.21.54.190:38013)
0811 17:18:17.525914 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:17.525934 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:17.525992 (+ 58us) server_negotiation.cc:407] Connection header received
0811 17:18:17.526875 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:17.526902 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:17.527376 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:17.527753 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:17.528827 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:17.530423 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:17.531418 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:17.532119 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:17.532288 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:17.535160 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:17.535178 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:17.535206 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:17.535245 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:17.537509 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:17.538122 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:17.538153 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:17.538161 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:17.538346 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:17.538768 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:17.538773 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:17.538775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:17.539235 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:17.539491 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:17.540286 (+ 795us) server_negotiation.cc:299] Negotiation successful
0811 17:18:17.540707 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":181,"threads_started":1}
I20260811 17:18:17.542407 11693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00a9a586c8254e499dfc2a3464727dba. Latest consensus state: current_term: 1 leader_uuid: "00a9a586c8254e499dfc2a3464727dba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a9a586c8254e499dfc2a3464727dba" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38013 } } }
I20260811 17:18:17.542630 11693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:17.542986 11710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:17.543478 11692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00a9a586c8254e499dfc2a3464727dba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00a9a586c8254e499dfc2a3464727dba" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 38013 } } }
I20260811 17:18:17.543571 11692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:17.546505 11710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:17.552870 11710 catalog_manager.cc:1347] Generated new cluster ID: 0e5a0677fce742b480a9da0a7c4ff183
I20260811 17:18:17.552956 11710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:17.559664 11710 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:17.560856 11710 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:17.568363 11710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba: Generated new TSK 0
I20260811 17:18:17.569199 11710 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.54.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
Entry for principal kudu/127.21.54.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab.
I20260811 17:18:17.607996 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:38013
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:33525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:17.731456 11716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:17.731734 11716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:17.731778 11716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:17.735945 11716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:17.736011 11716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:17.736104 11716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:18:17.741578 11716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:33525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.129.keytab
--principal=kudu/127.21.54.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:38013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:17.742754 11716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:17.743819 11716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:17.753824 11722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:17.753971 11721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:17.753824 11725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:17.754165 11716 server_base.cc:1034] running on GCE node
I20260811 17:18:17.754721 11716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:17.755370 11716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:17.756613 11716 hybrid_clock.cc:648] HybridClock initialized: now 1786468697756591 us; error 33 us; skew 500 ppm
Aug 11 17:18:17 dist-test-slave-fd7l krb5kdc[11590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468697, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:17.759421 11716 init.cc:377] Logged in from keytab as kudu/127.21.54.129@KRBTEST.COM (short username kudu)
I20260811 17:18:17.760411 11716 webserver.cc:460] Webserver started at http://127.21.54.129:45069/ using document root <none> and password file <none>
I20260811 17:18:17.761042 11716 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:17.761135 11716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:17.761368 11716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:17.763494 11716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "383b30213a2a462facc927f780c596ac"
format_stamp: "Formatted at 2026-08-11 17:18:17 on dist-test-slave-fd7l"
server_key: "982f1659e3a7200954bd759aae33e063"
server_key_iv: "90aa72e637898911457efdb07395f33e"
server_key_version: "encryptionkey@0"
I20260811 17:18:17.764101 11716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "383b30213a2a462facc927f780c596ac"
format_stamp: "Formatted at 2026-08-11 17:18:17 on dist-test-slave-fd7l"
server_key: "982f1659e3a7200954bd759aae33e063"
server_key_iv: "90aa72e637898911457efdb07395f33e"
server_key_version: "encryptionkey@0"
I20260811 17:18:17.768069 11716 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:18:17.770730 11734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:17.772090 11716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:18:17.772233 11716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "383b30213a2a462facc927f780c596ac"
format_stamp: "Formatted at 2026-08-11 17:18:17 on dist-test-slave-fd7l"
server_key: "982f1659e3a7200954bd759aae33e063"
server_key_iv: "90aa72e637898911457efdb07395f33e"
server_key_version: "encryptionkey@0"
I20260811 17:18:17.772354 11716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:17.786222 11716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:17.789520 11716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:17.789740 11716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:17.790350 11716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:18:17.791234 11716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:18:17.791308 11716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:17.791379 11716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:18:17.791421 11716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:17.801543 11716 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:35331
I20260811 17:18:17.801561 11865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:35331 every 8 connection(s)
I20260811 17:18:17.802629 11716 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:18:17.806495 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 11716
I20260811 17:18:17.806658 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:18:17.806995 21722 external_mini_cluster.cc:1428] Setting key b2053c73c98d0a237e975fb08419ca49
Aug 11 17:18:17 dist-test-slave-fd7l krb5kdc[11590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468697, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.129@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.54.130@KRBTEST.COM; defaulting to no policy
I20260811 17:18:17.818568 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:17.804644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:39965 (local address 127.21.54.190:38013)
0811 17:18:17.804807 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:17.804812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:17.805784 (+ 972us) server_negotiation.cc:407] Connection header received
0811 17:18:17.807555 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:17.807559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:17.807628 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:17.807734 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:17.809474 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:17.810009 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:17.811613 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:17.811864 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:17.811991 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:17.814619 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:17.814629 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:17.814632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:17.814659 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:17.816154 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:17.816805 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:17.816808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:17.816810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:17.816890 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:17.817292 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:17.817295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:17.817297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:17.817500 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:17.817616 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:17.818191 (+ 575us) server_negotiation.cc:299] Negotiation successful
0811 17:18:17.818331 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 17:18:17.819633 11868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:17.805098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38013 (local address 127.21.54.129:39965)
0811 17:18:17.805603 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:17.805661 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:17.807266 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:17.808120 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:17.808127 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:17.808591 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:17.809298 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:17.809323 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:17.810164 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:17.810171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:17.811476 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:17.811487 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:17.812036 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:17.812041 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:17.812225 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:17.812890 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:17.812914 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:17.814436 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:17.816289 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:17.816296 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:17.816314 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:17.816680 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:17.817009 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:17.817012 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:17.817014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:17.817171 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:17.817647 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:17.817654 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:17.817934 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 17:18:17.818133 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 17:18:17.818501 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":120,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.130@KRBTEST.COM" created.
I20260811 17:18:17.821116 11866 heartbeater.cc:347] Connected to a master server at 127.21.54.190:38013
I20260811 17:18:17.821393 11866 heartbeater.cc:464] Registering TS with master...
I20260811 17:18:17.821931 11866 heartbeater.cc:511] Master 127.21.54.190:38013 requested a full tablet report, sending...
I20260811 17:18:17.823743 11642 ts_manager.cc:194] Registered new tserver with Master: 383b30213a2a462facc927f780c596ac (127.21.54.129:35331)
I20260811 17:18:17.825639 11642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.129@KRBTEST.COM'} at 127.21.54.129:39965
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
Entry for principal kudu/127.21.54.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab.
I20260811 17:18:17.841152 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:38013
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:33525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:17.962029 11871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:17.962369 11871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:17.962416 11871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:17.966600 11871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:17.966682 11871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:17.966744 11871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:18:17.972371 11871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:33525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.130.keytab
--principal=kudu/127.21.54.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:38013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:17.973637 11871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:17.974678 11871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:17.985090 11876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:17.985146 11881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:17.985224 11877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:17.985311 11871 server_base.cc:1034] running on GCE node
I20260811 17:18:17.985926 11871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:17.986560 11871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:17.987757 11871 hybrid_clock.cc:648] HybridClock initialized: now 1786468697987737 us; error 29 us; skew 500 ppm
Aug 11 17:18:17 dist-test-slave-fd7l krb5kdc[11590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468697, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:17.990650 11871 init.cc:377] Logged in from keytab as kudu/127.21.54.130@KRBTEST.COM (short username kudu)
I20260811 17:18:17.991637 11871 webserver.cc:460] Webserver started at http://127.21.54.130:41255/ using document root <none> and password file <none>
I20260811 17:18:17.992235 11871 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:17.992331 11871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:17.992687 11871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:17.994802 11871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "a30f7d6e86024bb8afcfdf4a1bebc9bc"
format_stamp: "Formatted at 2026-08-11 17:18:17 on dist-test-slave-fd7l"
server_key: "544921bae1db42876b377b11c33d9950"
server_key_iv: "a49868d201d8f370dee52242d71e03cf"
server_key_version: "encryptionkey@0"
I20260811 17:18:17.995402 11871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "a30f7d6e86024bb8afcfdf4a1bebc9bc"
format_stamp: "Formatted at 2026-08-11 17:18:17 on dist-test-slave-fd7l"
server_key: "544921bae1db42876b377b11c33d9950"
server_key_iv: "a49868d201d8f370dee52242d71e03cf"
server_key_version: "encryptionkey@0"
I20260811 17:18:17.999588 11871 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 17:18:18.002341 11888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:18.003703 11871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:18:18.003847 11871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "a30f7d6e86024bb8afcfdf4a1bebc9bc"
format_stamp: "Formatted at 2026-08-11 17:18:17 on dist-test-slave-fd7l"
server_key: "544921bae1db42876b377b11c33d9950"
server_key_iv: "a49868d201d8f370dee52242d71e03cf"
server_key_version: "encryptionkey@0"
I20260811 17:18:18.003969 11871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:18.021359 11871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:18.024868 11871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:18.025115 11871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:18.025763 11871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:18:18.026644 11871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:18:18.026714 11871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:18.026787 11871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:18:18.026824 11871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:18.036330 11871 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:39737
I20260811 17:18:18.036361 12013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:39737 every 8 connection(s)
I20260811 17:18:18.037343 11871 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:18:18.039682 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 11871
I20260811 17:18:18.039820 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:18:18.040176 21722 external_mini_cluster.cc:1428] Setting key 7e630b90cbf168ad411d513be917b37a
Aug 11 17:18:18 dist-test-slave-fd7l krb5kdc[11590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468697, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.130@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.54.131@KRBTEST.COM; defaulting to no policy
I20260811 17:18:18.053864 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:18.039592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:40753 (local address 127.21.54.190:38013)
0811 17:18:18.039750 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:18.039759 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:18.040971 (+ 1212us) server_negotiation.cc:407] Connection header received
0811 17:18:18.042776 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:18.042778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:18.042822 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:18.042892 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:18.044578 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.045099 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.046410 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.046639 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.046777 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:18.049470 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:18.049477 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:18.049484 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:18.049523 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:18.051504 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:18.052148 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:18.052151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:18.052152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:18.052236 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:18.052643 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:18.052647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:18.052648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:18.052913 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:18.052998 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:18.053623 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 17:18:18.053734 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 17:18:18.054757 12016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:18.040011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38013 (local address 127.21.54.130:40753)
0811 17:18:18.040752 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:18.040844 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:18.042543 (+ 1699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:18.043135 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:18.043146 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:18.043670 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:18.044393 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:18.044409 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.045232 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.045235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:18.046282 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:18.046292 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.046811 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.046816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:18.047063 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:18.047694 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:18.047712 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:18.049279 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:18.051637 (+ 2358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:18.051643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:18.051656 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:18.052005 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:18.052326 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:18.052328 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:18.052330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:18.052481 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:18.053075 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:18.053085 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:18.053352 (+ 267us) client_negotiation.cc:769] Sending connection context
0811 17:18:18.053533 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 17:18:18.053811 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":167,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.131@KRBTEST.COM" created.
I20260811 17:18:18.056057 12014 heartbeater.cc:347] Connected to a master server at 127.21.54.190:38013
I20260811 17:18:18.056321 12014 heartbeater.cc:464] Registering TS with master...
I20260811 17:18:18.056916 12014 heartbeater.cc:511] Master 127.21.54.190:38013 requested a full tablet report, sending...
I20260811 17:18:18.058104 11642 ts_manager.cc:194] Registered new tserver with Master: a30f7d6e86024bb8afcfdf4a1bebc9bc (127.21.54.130:39737)
I20260811 17:18:18.058825 11642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.130@KRBTEST.COM'} at 127.21.54.130:40753
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
Entry for principal kudu/127.21.54.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab.
I20260811 17:18:18.073589 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:38013
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:33525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:18.190006 12019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:18.190348 12019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:18.190433 12019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:18.194583 12019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:18.194698 12019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:18.194800 12019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:18:18.200398 12019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:33525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.131.keytab
--principal=kudu/127.21.54.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:38013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:18.201653 12019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:18.202772 12019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:18.213346 12026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:18.213514 12025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:18.213630 12019 server_base.cc:1034] running on GCE node
W20260811 17:18:18.213346 12028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:18.214181 12019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:18.214918 12019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:18.216128 12019 hybrid_clock.cc:648] HybridClock initialized: now 1786468698216096 us; error 42 us; skew 500 ppm
Aug 11 17:18:18 dist-test-slave-fd7l krb5kdc[11590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468698, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:18.219230 12019 init.cc:377] Logged in from keytab as kudu/127.21.54.131@KRBTEST.COM (short username kudu)
I20260811 17:18:18.220156 12019 webserver.cc:460] Webserver started at http://127.21.54.131:37675/ using document root <none> and password file <none>
I20260811 17:18:18.220837 12019 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:18.220911 12019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:18.221102 12019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:18.223340 12019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "ad9bc820c05e42f8a796f7ab80b1f46f"
format_stamp: "Formatted at 2026-08-11 17:18:18 on dist-test-slave-fd7l"
server_key: "032cc873a9700646d9e7667c67f5383b"
server_key_iv: "2590c25e42df7cd4ad661b5f4b584de3"
server_key_version: "encryptionkey@0"
I20260811 17:18:18.223901 12019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "ad9bc820c05e42f8a796f7ab80b1f46f"
format_stamp: "Formatted at 2026-08-11 17:18:18 on dist-test-slave-fd7l"
server_key: "032cc873a9700646d9e7667c67f5383b"
server_key_iv: "2590c25e42df7cd4ad661b5f4b584de3"
server_key_version: "encryptionkey@0"
I20260811 17:18:18.228376 12019 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260811 17:18:18.231155 12035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:18.232687 12019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:18.232807 12019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "ad9bc820c05e42f8a796f7ab80b1f46f"
format_stamp: "Formatted at 2026-08-11 17:18:18 on dist-test-slave-fd7l"
server_key: "032cc873a9700646d9e7667c67f5383b"
server_key_iv: "2590c25e42df7cd4ad661b5f4b584de3"
server_key_version: "encryptionkey@0"
I20260811 17:18:18.232925 12019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:18.243247 12019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:18.246790 12019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:18.246945 12019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:18.247551 12019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:18:18.248369 12019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:18:18.248417 12019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:18.248457 12019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:18:18.248473 12019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:18.257793 12019 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:32943
I20260811 17:18:18.257855 12161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:32943 every 8 connection(s)
I20260811 17:18:18.258896 12019 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:18:18.262233 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 12019
I20260811 17:18:18.262387 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:18:18.262688 21722 external_mini_cluster.cc:1428] Setting key 2906e259835a2c6cf3cd4c564ddf1211
Aug 11 17:18:18 dist-test-slave-fd7l krb5kdc[11590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468698, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.131@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:18:18.274224 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:18.260962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:44141 (local address 127.21.54.190:38013)
0811 17:18:18.261126 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:18.261130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:18.261975 (+ 845us) server_negotiation.cc:407] Connection header received
0811 17:18:18.263672 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:18.263675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:18.263741 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:18.263843 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:18.265431 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.266005 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.267153 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.267386 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.267549 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:18.270322 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:18.270326 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:18.270328 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:18.270355 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:18.271883 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:18.272539 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:18.272543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:18.272545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:18.272638 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:18.273028 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:18.273033 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:18.273035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:18.273237 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:18.273373 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:18.273902 (+ 529us) server_negotiation.cc:299] Negotiation successful
0811 17:18:18.274038 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 17:18:18.275112 12164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:18.261353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:38013 (local address 127.21.54.131:44141)
0811 17:18:18.261811 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:18.261853 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:18.263357 (+ 1504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:18.264056 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:18.264066 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:18.264614 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:18.265295 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:18.265312 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.266128 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.266130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:18.267014 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:18.267024 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.267587 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.267590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:18.267744 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:18.268399 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:18.268417 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:18.270128 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:18.272032 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:18.272038 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:18.272057 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:18.272408 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:18.272748 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:18.272751 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:18.272753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:18.272926 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:18.273366 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:18.273373 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:18.273652 (+ 279us) client_negotiation.cc:769] Sending connection context
0811 17:18:18.273865 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 17:18:18.274235 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":128,"threads_started":1}
I20260811 17:18:18.276485 12162 heartbeater.cc:347] Connected to a master server at 127.21.54.190:38013
I20260811 17:18:18.276829 12162 heartbeater.cc:464] Registering TS with master...
I20260811 17:18:18.277456 12162 heartbeater.cc:511] Master 127.21.54.190:38013 requested a full tablet report, sending...
I20260811 17:18:18.278556 11642 ts_manager.cc:194] Registered new tserver with Master: ad9bc820c05e42f8a796f7ab80b1f46f (127.21.54.131:32943)
I20260811 17:18:18.279156 11642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.54.131@KRBTEST.COM'} at 127.21.54.131:44141
I20260811 17:18:18.289170 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:18:18.299623 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:18.291292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34672 (local address 127.21.54.190:38013)
0811 17:18:18.291449 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:18.291454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:18.291707 (+ 253us) server_negotiation.cc:407] Connection header received
0811 17:18:18.291861 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:18.291864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:18.291911 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:18.292003 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:18.292786 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.293309 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.294150 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.294381 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.294462 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:18.295997 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:18.296000 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:18.296002 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:18.296025 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:18.297428 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:18.298026 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:18.298035 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:18.298036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:18.298133 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:18.298527 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:18.298534 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:18.298536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:18.298827 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:18.299003 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:18.299278 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 17:18:18.299422 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:18:18.303253 11642 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:18:18.306577 11642 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:18:18.322530 12181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:18.316888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:35331 (local address 127.0.0.1:39442)
0811 17:18:18.317385 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:18.317402 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:18.317506 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:18.318156 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:18.318158 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:18.318184 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:18.318374 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:18.318382 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.319770 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.319773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:18.320890 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:18.320901 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.321976 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.321979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:18.322067 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:18.322103 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 17:18:18.322199 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 17:18:18.322335 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":29,"spinlock_wait_cycles":3200,"thread_start_us":90,"threads_started":1}
I20260811 17:18:18.322602 12184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:18.317329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39442 (local address 127.21.54.129:35331)
0811 17:18:18.317726 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:18.317739 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:18.317757 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:18:18.317818 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:18.317822 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:18.318027 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:18.318176 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:18.318507 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.319637 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.321062 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.321853 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.321970 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:18.322273 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:18.322339 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 17:18:18.322449 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":77,"threads_started":1}
I20260811 17:18:18.323287 12183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:18.316292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:32943 (local address 127.0.0.1:54868)
0811 17:18:18.317363 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:18.317376 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:18.317468 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:18.318147 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:18.318151 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:18.318179 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:18.318379 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:18.318384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.319535 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.319541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:18.321197 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:18.321209 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.322936 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.322940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:18.323033 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:18.323064 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 17:18:18.323126 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 17:18:18.323182 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1004,"mutex_wait_us":234,"spinlock_wait_cycles":10240,"thread_start_us":96,"threads_started":1}
I20260811 17:18:18.323386 12182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:18.316775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54868 (local address 127.21.54.131:32943)
0811 17:18:18.317215 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:18.317229 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:18.317563 (+ 334us) server_negotiation.cc:407] Connection header received
0811 17:18:18.317634 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:18.317638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:18.317809 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:18.317968 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:18.318522 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.319385 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.321335 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.322370 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.322500 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:18.323032 (+ 532us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:18.323156 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 17:18:18.323251 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":163,"threads_started":1}
I20260811 17:18:18.325536 11788 tablet_service.cc:1467] Processing CreateTablet for tablet a8c2544958f34365b4331b4e0a379897 (DEFAULT_TABLE table=test-table [id=b428ae0e3c614ec5bbc4ea19d348dbb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:18:18.325726 12090 tablet_service.cc:1467] Processing CreateTablet for tablet a8c2544958f34365b4331b4e0a379897 (DEFAULT_TABLE table=test-table [id=b428ae0e3c614ec5bbc4ea19d348dbb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:18:18.326622 11788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8c2544958f34365b4331b4e0a379897. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:18.326737 12090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8c2544958f34365b4331b4e0a379897. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:18.327797 12177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:18.316008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:39737 (local address 127.0.0.1:47874)
0811 17:18:18.316961 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:18.317015 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:18.317148 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:18.317974 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:18.317980 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:18.318004 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:18.318160 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:18.318168 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.319665 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.319669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:18.321197 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:18.321209 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.327355 (+ 6146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.327358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:18.327464 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:18.327488 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 17:18:18.327574 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 17:18:18.327660 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"mutex_wait_us":125,"thread_start_us":141,"threads_started":1}
I20260811 17:18:18.328058 12179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:18.316176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47874 (local address 127.21.54.130:39737)
0811 17:18:18.317464 (+ 1288us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:18.317476 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:18.317503 (+ 27us) server_negotiation.cc:407] Connection header received
0811 17:18:18.317597 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:18.317603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:18.317809 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:18.317969 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:18.318300 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.319506 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.326139 (+ 6633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.327024 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.327146 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:18.327286 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:18.327646 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 17:18:18.327753 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1180,"thread_start_us":189,"threads_started":1}
I20260811 17:18:18.331146 11944 tablet_service.cc:1467] Processing CreateTablet for tablet a8c2544958f34365b4331b4e0a379897 (DEFAULT_TABLE table=test-table [id=b428ae0e3c614ec5bbc4ea19d348dbb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:18:18.332261 11944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8c2544958f34365b4331b4e0a379897. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:18.333415 12185 tablet_bootstrap.cc:492] T a8c2544958f34365b4331b4e0a379897 P 383b30213a2a462facc927f780c596ac: Bootstrap starting.
I20260811 17:18:18.334728 12186 tablet_bootstrap.cc:492] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f: Bootstrap starting.
I20260811 17:18:18.335644 12185 tablet_bootstrap.cc:654] T a8c2544958f34365b4331b4e0a379897 P 383b30213a2a462facc927f780c596ac: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:18.336445 12185 log.cc:824] T a8c2544958f34365b4331b4e0a379897 P 383b30213a2a462facc927f780c596ac: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:18.336911 12186 tablet_bootstrap.cc:654] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:18.338152 12186 log.cc:824] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:18.338754 12185 tablet_bootstrap.cc:492] T a8c2544958f34365b4331b4e0a379897 P 383b30213a2a462facc927f780c596ac: No bootstrap required, opened a new log
I20260811 17:18:18.338972 12185 ts_tablet_manager.cc:1397] T a8c2544958f34365b4331b4e0a379897 P 383b30213a2a462facc927f780c596ac: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260811 17:18:18.339998 12189 tablet_bootstrap.cc:492] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc: Bootstrap starting.
I20260811 17:18:18.340207 12186 tablet_bootstrap.cc:492] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f: No bootstrap required, opened a new log
I20260811 17:18:18.340440 12186 ts_tablet_manager.cc:1397] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 17:18:18.342576 12189 tablet_bootstrap.cc:654] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:18.343467 12186 raft_consensus.cc:348] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30f7d6e86024bb8afcfdf4a1bebc9bc" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39737 } } peers { permanent_uuid: "383b30213a2a462facc927f780c596ac" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35331 } } peers { permanent_uuid: "ad9bc820c05e42f8a796f7ab80b1f46f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 32943 } }
I20260811 17:18:18.343709 12186 raft_consensus.cc:374] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:18.343559 12185 raft_consensus.cc:348] T a8c2544958f34365b4331b4e0a379897 P 383b30213a2a462facc927f780c596ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30f7d6e86024bb8afcfdf4a1bebc9bc" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39737 } } peers { permanent_uuid: "383b30213a2a462facc927f780c596ac" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35331 } } peers { permanent_uuid: "ad9bc820c05e42f8a796f7ab80b1f46f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 32943 } }
I20260811 17:18:18.343773 12186 raft_consensus.cc:729] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9bc820c05e42f8a796f7ab80b1f46f, State: Initialized, Role: FOLLOWER
I20260811 17:18:18.343828 12185 raft_consensus.cc:374] T a8c2544958f34365b4331b4e0a379897 P 383b30213a2a462facc927f780c596ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:18.343884 12185 raft_consensus.cc:729] T a8c2544958f34365b4331b4e0a379897 P 383b30213a2a462facc927f780c596ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 383b30213a2a462facc927f780c596ac, State: Initialized, Role: FOLLOWER
I20260811 17:18:18.344314 12189 log.cc:824] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:18.344254 12186 consensus_queue.cc:260] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30f7d6e86024bb8afcfdf4a1bebc9bc" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39737 } } peers { permanent_uuid: "383b30213a2a462facc927f780c596ac" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35331 } } peers { permanent_uuid: "ad9bc820c05e42f8a796f7ab80b1f46f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 32943 } }
I20260811 17:18:18.344302 12185 consensus_queue.cc:260] T a8c2544958f34365b4331b4e0a379897 P 383b30213a2a462facc927f780c596ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30f7d6e86024bb8afcfdf4a1bebc9bc" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39737 } } peers { permanent_uuid: "383b30213a2a462facc927f780c596ac" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35331 } } peers { permanent_uuid: "ad9bc820c05e42f8a796f7ab80b1f46f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 32943 } }
I20260811 17:18:18.345194 12185 ts_tablet_manager.cc:1428] T a8c2544958f34365b4331b4e0a379897 P 383b30213a2a462facc927f780c596ac: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 17:18:18.345220 12186 ts_tablet_manager.cc:1428] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260811 17:18:18.345546 11866 heartbeater.cc:503] Master 127.21.54.190:38013 was elected leader, sending a full tablet report...
I20260811 17:18:18.345571 12162 heartbeater.cc:503] Master 127.21.54.190:38013 was elected leader, sending a full tablet report...
I20260811 17:18:18.347374 12189 tablet_bootstrap.cc:492] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc: No bootstrap required, opened a new log
I20260811 17:18:18.347616 12189 ts_tablet_manager.cc:1397] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 17:18:18.351493 12189 raft_consensus.cc:348] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30f7d6e86024bb8afcfdf4a1bebc9bc" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39737 } } peers { permanent_uuid: "383b30213a2a462facc927f780c596ac" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35331 } } peers { permanent_uuid: "ad9bc820c05e42f8a796f7ab80b1f46f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 32943 } }
I20260811 17:18:18.351708 12189 raft_consensus.cc:374] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:18.351742 12189 raft_consensus.cc:729] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a30f7d6e86024bb8afcfdf4a1bebc9bc, State: Initialized, Role: FOLLOWER
I20260811 17:18:18.352141 12189 consensus_queue.cc:260] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30f7d6e86024bb8afcfdf4a1bebc9bc" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39737 } } peers { permanent_uuid: "383b30213a2a462facc927f780c596ac" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35331 } } peers { permanent_uuid: "ad9bc820c05e42f8a796f7ab80b1f46f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 32943 } }
I20260811 17:18:18.352878 12014 heartbeater.cc:503] Master 127.21.54.190:38013 was elected leader, sending a full tablet report...
I20260811 17:18:18.353229 12189 ts_tablet_manager.cc:1428] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 17:18:18.435122 12190 raft_consensus.cc:482] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:18:18.435407 12190 raft_consensus.cc:504] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30f7d6e86024bb8afcfdf4a1bebc9bc" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39737 } } peers { permanent_uuid: "383b30213a2a462facc927f780c596ac" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35331 } } peers { permanent_uuid: "ad9bc820c05e42f8a796f7ab80b1f46f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 32943 } }
I20260811 17:18:18.436995 12190 leader_election.cc:288] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a30f7d6e86024bb8afcfdf4a1bebc9bc (127.21.54.130:39737), 383b30213a2a462facc927f780c596ac (127.21.54.129:35331)
I20260811 17:18:18.442054 12179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:18.437297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:34051 (local address 127.21.54.130:39737)
0811 17:18:18.437483 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:18.437493 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:18.437515 (+ 22us) server_negotiation.cc:407] Connection header received
0811 17:18:18.437586 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:18.437590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:18.437677 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:18.438138 (+ 461us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:18.438341 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.439055 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.440815 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.441493 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.441625 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:18.441685 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:18.441855 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 17:18:18.441924 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 17:18:18.442054 12164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:18.437209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:39737 (local address 127.21.54.131:34051)
0811 17:18:18.437366 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:18.437384 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:18.437493 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:18.437818 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:18.437822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:18.437846 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:18.438008 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:18.438019 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.439159 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.439165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:18.440671 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:18.440684 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.441614 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.441618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:18.441723 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:18.441760 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 17:18:18.441841 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 17:18:18.441916 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 17:18:18.442847 12194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:18.437492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:35331 (local address 127.21.54.131:58843)
0811 17:18:18.437886 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:18.437901 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:18.438003 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:18.438822 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:18.438825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:18.438843 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:18.438961 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:18.438966 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.439995 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.439999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:18.441257 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:18.441269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.442428 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.442431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:18.442540 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:18.442565 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 17:18:18.442642 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 17:18:18.442718 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":181,"threads_started":1}
I20260811 17:18:18.442929 12184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:18.437678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:58843 (local address 127.21.54.129:35331)
0811 17:18:18.437938 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:18.437946 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:18.437964 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:18:18.438125 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:18.438128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:18.438633 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:18.438715 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:18.439160 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.439868 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.441396 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.442295 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.442469 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:18.442544 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:18.442685 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 17:18:18.442775 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260811 17:18:18.442987 11964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c2544958f34365b4331b4e0a379897" candidate_uuid: "ad9bc820c05e42f8a796f7ab80b1f46f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30f7d6e86024bb8afcfdf4a1bebc9bc" is_pre_election: true
I20260811 17:18:18.443348 11964 raft_consensus.cc:2454] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9bc820c05e42f8a796f7ab80b1f46f in term 0.
I20260811 17:18:18.443522 11812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c2544958f34365b4331b4e0a379897" candidate_uuid: "ad9bc820c05e42f8a796f7ab80b1f46f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "383b30213a2a462facc927f780c596ac" is_pre_election: true
I20260811 17:18:18.443818 11812 raft_consensus.cc:2454] T a8c2544958f34365b4331b4e0a379897 P 383b30213a2a462facc927f780c596ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9bc820c05e42f8a796f7ab80b1f46f in term 0.
I20260811 17:18:18.444042 12038 leader_election.cc:302] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a30f7d6e86024bb8afcfdf4a1bebc9bc, ad9bc820c05e42f8a796f7ab80b1f46f; no voters:
I20260811 17:18:18.444324 12190 raft_consensus.cc:2781] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:18:18.444403 12190 raft_consensus.cc:482] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:18:18.444464 12190 raft_consensus.cc:3037] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:18.445976 12190 raft_consensus.cc:504] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30f7d6e86024bb8afcfdf4a1bebc9bc" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39737 } } peers { permanent_uuid: "383b30213a2a462facc927f780c596ac" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35331 } } peers { permanent_uuid: "ad9bc820c05e42f8a796f7ab80b1f46f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 32943 } }
I20260811 17:18:18.446374 12190 leader_election.cc:288] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [CANDIDATE]: Term 1 election: Requested vote from peers a30f7d6e86024bb8afcfdf4a1bebc9bc (127.21.54.130:39737), 383b30213a2a462facc927f780c596ac (127.21.54.129:35331)
I20260811 17:18:18.446834 11964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c2544958f34365b4331b4e0a379897" candidate_uuid: "ad9bc820c05e42f8a796f7ab80b1f46f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30f7d6e86024bb8afcfdf4a1bebc9bc"
I20260811 17:18:18.446943 11812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c2544958f34365b4331b4e0a379897" candidate_uuid: "ad9bc820c05e42f8a796f7ab80b1f46f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "383b30213a2a462facc927f780c596ac"
I20260811 17:18:18.446986 11964 raft_consensus.cc:3037] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:18.447100 11812 raft_consensus.cc:3037] T a8c2544958f34365b4331b4e0a379897 P 383b30213a2a462facc927f780c596ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:18.448585 11964 raft_consensus.cc:2454] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9bc820c05e42f8a796f7ab80b1f46f in term 1.
I20260811 17:18:18.448818 11812 raft_consensus.cc:2454] T a8c2544958f34365b4331b4e0a379897 P 383b30213a2a462facc927f780c596ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9bc820c05e42f8a796f7ab80b1f46f in term 1.
I20260811 17:18:18.448905 12038 leader_election.cc:302] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a30f7d6e86024bb8afcfdf4a1bebc9bc, ad9bc820c05e42f8a796f7ab80b1f46f; no voters:
I20260811 17:18:18.449225 12190 raft_consensus.cc:2781] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:18.449466 12190 raft_consensus.cc:686] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [term 1 LEADER]: Becoming Leader. State: Replica: ad9bc820c05e42f8a796f7ab80b1f46f, State: Running, Role: LEADER
I20260811 17:18:18.449846 12190 consensus_queue.cc:237] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30f7d6e86024bb8afcfdf4a1bebc9bc" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39737 } } peers { permanent_uuid: "383b30213a2a462facc927f780c596ac" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35331 } } peers { permanent_uuid: "ad9bc820c05e42f8a796f7ab80b1f46f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 32943 } }
I20260811 17:18:18.453218 11641 catalog_manager.cc:5495] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f reported cstate change: term changed from 0 to 1, leader changed from <none> to ad9bc820c05e42f8a796f7ab80b1f46f (127.21.54.131). New cstate: current_term: 1 leader_uuid: "ad9bc820c05e42f8a796f7ab80b1f46f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30f7d6e86024bb8afcfdf4a1bebc9bc" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "383b30213a2a462facc927f780c596ac" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9bc820c05e42f8a796f7ab80b1f46f" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 32943 } health_report { overall_health: HEALTHY } } }
I20260811 17:18:18.475373 12182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:18.471854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54884 (local address 127.21.54.131:32943)
0811 17:18:18.471989 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:18.471993 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:18.472006 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:18:18.472086 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:18.472088 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:18.472154 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:18.472246 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:18:18.472473 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.473031 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.473985 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.474206 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.474287 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:18.474565 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:18:18.474998 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:18:18.475091 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:18.475176 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 17:18:18.475229 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 17:18:18.482395 11812 raft_consensus.cc:1261] T a8c2544958f34365b4331b4e0a379897 P 383b30213a2a462facc927f780c596ac [term 1 FOLLOWER]: Refusing update from remote peer ad9bc820c05e42f8a796f7ab80b1f46f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:18:18.482458 11964 raft_consensus.cc:1261] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc [term 1 FOLLOWER]: Refusing update from remote peer ad9bc820c05e42f8a796f7ab80b1f46f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:18:18.483243 12190 consensus_queue.cc:1035] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [LEADER]: Connected to new peer: Peer: permanent_uuid: "383b30213a2a462facc927f780c596ac" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 35331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:18:18.483498 12195 consensus_queue.cc:1035] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a30f7d6e86024bb8afcfdf4a1bebc9bc" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 39737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:18:18.491030 12198 mvcc.cc:205] Tried to move back new op lower bound from 7317375788974608384 to 7317375788853309440. Current Snapshot: MvccSnapshot[applied={T|T < 7317375788974608384}]
I20260811 17:18:18.494043 12201 mvcc.cc:205] Tried to move back new op lower bound from 7317375788974608384 to 7317375788853309440. Current Snapshot: MvccSnapshot[applied={T|T < 7317375788974608384}]
I20260811 17:18:18.494313 12200 mvcc.cc:205] Tried to move back new op lower bound from 7317375788974608384 to 7317375788853309440. Current Snapshot: MvccSnapshot[applied={T|T < 7317375788974608384}]
I20260811 17:18:18.500643 11642 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:18:18.500836 11642 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:18:18.503443 11642 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00a9a586c8254e499dfc2a3464727dba: Sending DeleteTablet for 3 replicas of tablet a8c2544958f34365b4331b4e0a379897
I20260811 17:18:18.504400 12090 tablet_service.cc:1514] Processing DeleteTablet for tablet a8c2544958f34365b4331b4e0a379897 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:18:18 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:54868
I20260811 17:18:18.504433 11944 tablet_service.cc:1514] Processing DeleteTablet for tablet a8c2544958f34365b4331b4e0a379897 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:18:18 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:47874
I20260811 17:18:18.504647 11788 tablet_service.cc:1514] Processing DeleteTablet for tablet a8c2544958f34365b4331b4e0a379897 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:18:18 UTC) from {username='kudu', principal='kudu/127.21.54.190@KRBTEST.COM'} at 127.0.0.1:39442
I20260811 17:18:18.505057 12210 tablet_replica.cc:330] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc: stopping tablet replica
I20260811 17:18:18.505409 12210 raft_consensus.cc:2229] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:18:18.505601 12209 tablet_replica.cc:330] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f: stopping tablet replica
I20260811 17:18:18.505761 12210 raft_consensus.cc:2258] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:18:18.505865 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 11716
I20260811 17:18:18.506124 12209 raft_consensus.cc:2229] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:18:18.506590 12209 raft_consensus.cc:2258] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:18:18.508922 12209 ts_tablet_manager.cc:1905] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:18:18.509811 12210 ts_tablet_manager.cc:1905] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:18:18.511809 12209 ts_tablet_manager.cc:1918] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:18:18.511894 12209 log.cc:1196] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/a8c2544958f34365b4331b4e0a379897
I20260811 17:18:18.512236 12209 ts_tablet_manager.cc:1939] T a8c2544958f34365b4331b4e0a379897 P ad9bc820c05e42f8a796f7ab80b1f46f: Deleting consensus metadata
I20260811 17:18:18.513695 11627 catalog_manager.cc:4841] TS ad9bc820c05e42f8a796f7ab80b1f46f (127.21.54.131:32943): tablet a8c2544958f34365b4331b4e0a379897 (table test-table [id=b428ae0e3c614ec5bbc4ea19d348dbb5]) successfully deleted
I20260811 17:18:18.520897 12210 ts_tablet_manager.cc:1918] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:18:18.521056 12210 log.cc:1196] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/a8c2544958f34365b4331b4e0a379897
I20260811 17:18:18.521418 12210 ts_tablet_manager.cc:1939] T a8c2544958f34365b4331b4e0a379897 P a30f7d6e86024bb8afcfdf4a1bebc9bc: Deleting consensus metadata
I20260811 17:18:18.522984 11626 catalog_manager.cc:4841] TS a30f7d6e86024bb8afcfdf4a1bebc9bc (127.21.54.130:39737): tablet a8c2544958f34365b4331b4e0a379897 (table test-table [id=b428ae0e3c614ec5bbc4ea19d348dbb5]) successfully deleted
I20260811 17:18:18.528494 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 11871
W20260811 17:18:18.528451 11626 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.54.129:35331 (error 108)
I20260811 17:18:18.534461 12177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:18.533930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:35331 (local address 127.0.0.1:39452)
0811 17:18:18.534131 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:18.534342 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.129:35331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260811 17:18:18.534855 11626 catalog_manager.cc:4584] TS 383b30213a2a462facc927f780c596ac (127.21.54.129:35331): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a8c2544958f34365b4331b4e0a379897: Network error: Client connection negotiation failed: client connection to 127.21.54.129:35331: connect: Connection refused (error 111)
I20260811 17:18:18.540647 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 12019
I20260811 17:18:18.551525 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 11606
2026-08-11T17:18:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2608 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T17:18:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:18Z Disabled control of system clock
I20260811 17:18:18.597168 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:39683
--webserver_interface=127.21.54.190
--webserver_port=0
--builtin_ntp_servers=127.21.54.148:35407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:39683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 17:18:18.722657 12218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:18.722965 12218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:18.723002 12218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:18.727555 12218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:18.727630 12218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:18.727648 12218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:18.727666 12218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:18.727681 12218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:18.732748 12218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:35407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:39683
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:39683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:18.733906 12218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:18.735348 12218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:18.744397 12224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:18.744406 12225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:18.744567 12227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:18.744920 12218 server_base.cc:1034] running on GCE node
I20260811 17:18:18.745482 12218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:18.746687 12218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:18.747884 12218 hybrid_clock.cc:648] HybridClock initialized: now 1786468698747827 us; error 67 us; skew 500 ppm
I20260811 17:18:18.755302 12218 webserver.cc:460] Webserver started at http://127.21.54.190:40961/ using document root <none> and password file <none>
I20260811 17:18:18.755964 12218 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:18.756031 12218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:18.756227 12218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:18.758608 12218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "6528593d1ea04e83ae4bce8090e98ed1"
format_stamp: "Formatted at 2026-08-11 17:18:18 on dist-test-slave-fd7l"
server_key: "338f32eb5c43ebd503b243e4adca727d"
server_key_iv: "1e9fd02291e08509fca4020f9e2be892"
server_key_version: "encryptionkey@0"
I20260811 17:18:18.759200 12218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "6528593d1ea04e83ae4bce8090e98ed1"
format_stamp: "Formatted at 2026-08-11 17:18:18 on dist-test-slave-fd7l"
server_key: "338f32eb5c43ebd503b243e4adca727d"
server_key_iv: "1e9fd02291e08509fca4020f9e2be892"
server_key_version: "encryptionkey@0"
I20260811 17:18:18.763113 12218 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 17:18:18.765704 12233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:18.767063 12218 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:18.767248 12218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "6528593d1ea04e83ae4bce8090e98ed1"
format_stamp: "Formatted at 2026-08-11 17:18:18 on dist-test-slave-fd7l"
server_key: "338f32eb5c43ebd503b243e4adca727d"
server_key_iv: "1e9fd02291e08509fca4020f9e2be892"
server_key_version: "encryptionkey@0"
I20260811 17:18:18.767375 12218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:18.786652 12218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:18.787410 12218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:18.787632 12218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:18.795298 12218 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:39683
I20260811 17:18:18.795351 12292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:39683 every 8 connection(s)
I20260811 17:18:18.796362 12218 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:18.799827 12293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:18.805758 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 12218
I20260811 17:18:18.805900 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:18.805992 12293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1: Bootstrap starting.
I20260811 17:18:18.806228 21722 external_mini_cluster.cc:1428] Setting key 19a518c17669c1ff299869ce87e05857
I20260811 17:18:18.808844 12293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:18.809798 12293 log.cc:824] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:18.812248 12293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1: No bootstrap required, opened a new log
I20260811 17:18:18.816851 12293 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6528593d1ea04e83ae4bce8090e98ed1" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39683 } }
I20260811 17:18:18.817142 12293 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:18.817191 12293 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6528593d1ea04e83ae4bce8090e98ed1, State: Initialized, Role: FOLLOWER
I20260811 17:18:18.817718 12296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:18.807985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42064 (local address 127.21.54.190:39683)
0811 17:18:18.808776 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:18.808800 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:18.808847 (+ 47us) server_negotiation.cc:407] Connection header received
0811 17:18:18.809753 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:18.809779 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:18.810309 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:18.810616 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:18.811086 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.812581 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.813592 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:18.814335 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:18.814513 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:18.815427 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:18.815434 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:18.815454 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:18.816058 (+ 604us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:18.816129 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:18.816155 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:18.816259 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:18.816421 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 17:18:18.816736 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":153,"threads_started":1}
I20260811 17:18:18.817667 12293 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6528593d1ea04e83ae4bce8090e98ed1" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39683 } }
I20260811 17:18:18.817818 12293 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:18.817885 12293 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:18.817984 12293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:18.818950 12293 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6528593d1ea04e83ae4bce8090e98ed1" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39683 } }
I20260811 17:18:18.819305 12293 leader_election.cc:302] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6528593d1ea04e83ae4bce8090e98ed1; no voters:
I20260811 17:18:18.819624 12293 leader_election.cc:288] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:18.819734 12298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:18.819988 12298 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1 [term 1 LEADER]: Becoming Leader. State: Replica: 6528593d1ea04e83ae4bce8090e98ed1, State: Running, Role: LEADER
I20260811 17:18:18.820338 12298 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6528593d1ea04e83ae4bce8090e98ed1" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39683 } }
I20260811 17:18:18.821022 12293 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:18.821535 12301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6528593d1ea04e83ae4bce8090e98ed1. Latest consensus state: current_term: 1 leader_uuid: "6528593d1ea04e83ae4bce8090e98ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6528593d1ea04e83ae4bce8090e98ed1" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39683 } } }
I20260811 17:18:18.821722 12301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:18.821995 12299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6528593d1ea04e83ae4bce8090e98ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6528593d1ea04e83ae4bce8090e98ed1" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 39683 } } }
I20260811 17:18:18.822090 12299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:18.822176 12305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:18.825770 12305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:18.832049 12305 catalog_manager.cc:1347] Generated new cluster ID: eb846c6929654f97a9b415c35cf19fb2
I20260811 17:18:18.832142 12305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:18.842190 12305 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:18.843501 12305 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:18.850015 12305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1: Generated new TSK 0
I20260811 17:18:18.850826 12305 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:18.861318 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.129:0
--local_ip_for_outbound_sockets=127.21.54.129
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:39683
--builtin_ntp_servers=127.21.54.148:35407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 17:18:18.982597 12322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:18.982944 12322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:18.983023 12322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:18.987691 12322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:18.987803 12322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:18.987911 12322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.129
I20260811 17:18:18.993175 12322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:35407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.54.129
--webserver_port=0
--tserver_master_addrs=127.21.54.190:39683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.129
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:18.994372 12322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:18.995820 12322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:19.006371 12328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:19.006361 12332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:19.006335 12329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:19.006652 12322 server_base.cc:1034] running on GCE node
I20260811 17:18:19.007153 12322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:19.007889 12322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:19.009110 12322 hybrid_clock.cc:648] HybridClock initialized: now 1786468699009041 us; error 86 us; skew 500 ppm
I20260811 17:18:19.011135 12322 webserver.cc:460] Webserver started at http://127.21.54.129:39899/ using document root <none> and password file <none>
I20260811 17:18:19.011751 12322 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:19.011848 12322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:19.012104 12322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:19.014371 12322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-0/data/instance:
uuid: "9e1d8776265c4f92b24c1dde6489b69c"
format_stamp: "Formatted at 2026-08-11 17:18:19 on dist-test-slave-fd7l"
server_key: "49136c0e6b895b36395102255b65faab"
server_key_iv: "a530d9f5f2d413f6bc6e1e80e4ca6969"
server_key_version: "encryptionkey@0"
I20260811 17:18:19.014964 12322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance:
uuid: "9e1d8776265c4f92b24c1dde6489b69c"
format_stamp: "Formatted at 2026-08-11 17:18:19 on dist-test-slave-fd7l"
server_key: "49136c0e6b895b36395102255b65faab"
server_key_iv: "a530d9f5f2d413f6bc6e1e80e4ca6969"
server_key_version: "encryptionkey@0"
I20260811 17:18:19.018846 12322 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 17:18:19.021533 12338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:19.023140 12322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:19.023278 12322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-0/wal
uuid: "9e1d8776265c4f92b24c1dde6489b69c"
format_stamp: "Formatted at 2026-08-11 17:18:19 on dist-test-slave-fd7l"
server_key: "49136c0e6b895b36395102255b65faab"
server_key_iv: "a530d9f5f2d413f6bc6e1e80e4ca6969"
server_key_version: "encryptionkey@0"
I20260811 17:18:19.023397 12322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:19.037319 12322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:19.038070 12322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:19.038278 12322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:19.038913 12322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:18:19.039983 12322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:18:19.040066 12322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:19.040144 12322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:18:19.040187 12322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:19.050061 12322 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.129:37793
I20260811 17:18:19.050083 12470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.129:37793 every 8 connection(s)
I20260811 17:18:19.051011 12322 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-0/data/info.pb
I20260811 17:18:19.059732 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 12322
I20260811 17:18:19.059926 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-0/wal/instance
I20260811 17:18:19.060308 21722 external_mini_cluster.cc:1428] Setting key 6339462441a3711c137b280f714fd081
I20260811 17:18:19.061275 12296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:19.053054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:51789 (local address 127.21.54.190:39683)
0811 17:18:19.053202 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:19.053206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:19.054365 (+ 1159us) server_negotiation.cc:407] Connection header received
0811 17:18:19.055797 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:19.055800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:19.055852 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:19.055935 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:19.057003 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.057625 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.058789 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.059050 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.059208 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:19.060735 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:19.060739 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:19.060742 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:19.060763 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:19.060785 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:19.060794 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:19.060877 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:19.061055 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 17:18:19.061125 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 17:18:19.062263 12474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:19.053511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:39683 (local address 127.21.54.129:51789)
0811 17:18:19.054193 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:19.054235 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:19.055580 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:19.056108 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:19.056114 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:19.056195 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:19.056808 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:19.056824 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.057758 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.057760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:19.058643 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:19.058653 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.059272 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.059276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:19.059478 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:19.060315 (+ 837us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:18:19.060316 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:19.060395 (+ 79us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:18:19.060499 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:19.060880 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:19.060893 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:19.060900 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 17:18:19.061053 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 17:18:19.061349 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":130,"threads_started":1}
I20260811 17:18:19.062767 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.130:0
--local_ip_for_outbound_sockets=127.21.54.130
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:39683
--builtin_ntp_servers=127.21.54.148:35407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 17:18:19.063606 12471 heartbeater.cc:347] Connected to a master server at 127.21.54.190:39683
I20260811 17:18:19.063880 12471 heartbeater.cc:464] Registering TS with master...
I20260811 17:18:19.064616 12471 heartbeater.cc:511] Master 127.21.54.190:39683 requested a full tablet report, sending...
I20260811 17:18:19.066357 12251 ts_manager.cc:194] Registered new tserver with Master: 9e1d8776265c4f92b24c1dde6489b69c (127.21.54.129:37793)
I20260811 17:18:19.068226 12251 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.54.129:51789
W20260811 17:18:19.184547 12475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:19.184841 12475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:19.184881 12475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:19.188912 12475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:19.188970 12475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:19.189028 12475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.130
I20260811 17:18:19.193826 12475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:35407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.54.130
--webserver_port=0
--tserver_master_addrs=127.21.54.190:39683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.130
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:19.194927 12475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:19.196326 12475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:19.206779 12481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:19.206837 12480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:19.206837 12475 server_base.cc:1034] running on GCE node
W20260811 17:18:19.207028 12485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:19.207455 12475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:19.208091 12475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:19.209266 12475 hybrid_clock.cc:648] HybridClock initialized: now 1786468699209229 us; error 47 us; skew 500 ppm
I20260811 17:18:19.211369 12475 webserver.cc:460] Webserver started at http://127.21.54.130:40305/ using document root <none> and password file <none>
I20260811 17:18:19.211997 12475 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:19.212097 12475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:19.212354 12475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:19.214599 12475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-1/data/instance:
uuid: "5e403d38fad447389a39eeb605993ad4"
format_stamp: "Formatted at 2026-08-11 17:18:19 on dist-test-slave-fd7l"
server_key: "067aee4af8d6e4b7ca658aed9f3190e8"
server_key_iv: "ac77d44a10253bb7e2c7e5b26f033c74"
server_key_version: "encryptionkey@0"
I20260811 17:18:19.215193 12475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance:
uuid: "5e403d38fad447389a39eeb605993ad4"
format_stamp: "Formatted at 2026-08-11 17:18:19 on dist-test-slave-fd7l"
server_key: "067aee4af8d6e4b7ca658aed9f3190e8"
server_key_iv: "ac77d44a10253bb7e2c7e5b26f033c74"
server_key_version: "encryptionkey@0"
I20260811 17:18:19.219127 12475 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:18:19.221942 12492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:19.223646 12475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:18:19.223807 12475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-1/wal
uuid: "5e403d38fad447389a39eeb605993ad4"
format_stamp: "Formatted at 2026-08-11 17:18:19 on dist-test-slave-fd7l"
server_key: "067aee4af8d6e4b7ca658aed9f3190e8"
server_key_iv: "ac77d44a10253bb7e2c7e5b26f033c74"
server_key_version: "encryptionkey@0"
I20260811 17:18:19.223971 12475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:19.239212 12475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:19.239969 12475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:19.240181 12475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:19.240862 12475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:18:19.241935 12475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:18:19.242014 12475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:19.242089 12475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:18:19.242131 12475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:19.251317 12475 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.130:38415
I20260811 17:18:19.251339 12622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.130:38415 every 8 connection(s)
I20260811 17:18:19.252358 12475 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-1/data/info.pb
I20260811 17:18:19.262362 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 12475
I20260811 17:18:19.262493 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-1/wal/instance
I20260811 17:18:19.262856 21722 external_mini_cluster.cc:1428] Setting key 2c50c460d2fcce9de04fa0c7b51bbac2
I20260811 17:18:19.262946 12296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:19.254528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.130:52669 (local address 127.21.54.190:39683)
0811 17:18:19.254683 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:19.254687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:19.255623 (+ 936us) server_negotiation.cc:407] Connection header received
0811 17:18:19.257205 (+ 1582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:19.257210 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:19.257276 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:19.257361 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:19.258343 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.258895 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.260295 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.260600 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.260715 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:19.262126 (+ 1411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:19.262130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:19.262134 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:19.262164 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:19.262191 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:19.262204 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:19.262310 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:19.262700 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 17:18:19.262781 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 17:18:19.264178 12625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:19.254708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:39683 (local address 127.21.54.130:52669)
0811 17:18:19.255396 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:19.255490 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:19.256951 (+ 1461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:19.257574 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:19.257581 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:19.257668 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:19.258182 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:19.258194 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.259045 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.259055 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:19.260158 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:19.260169 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.260913 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.260917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:19.261081 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:19.261761 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:18:19.261763 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:19.261805 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:18:19.261877 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:19.262337 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:19.262347 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:19.262353 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 17:18:19.262655 (+ 302us) client_negotiation.cc:240] Negotiation successful
0811 17:18:19.262986 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":119,"threads_started":1}
I20260811 17:18:19.265533 12623 heartbeater.cc:347] Connected to a master server at 127.21.54.190:39683
I20260811 17:18:19.265661 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.54.131:0
--local_ip_for_outbound_sockets=127.21.54.131
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:39683
--builtin_ntp_servers=127.21.54.148:35407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 17:18:19.265836 12623 heartbeater.cc:464] Registering TS with master...
I20260811 17:18:19.266408 12623 heartbeater.cc:511] Master 127.21.54.190:39683 requested a full tablet report, sending...
I20260811 17:18:19.267657 12251 ts_manager.cc:194] Registered new tserver with Master: 5e403d38fad447389a39eeb605993ad4 (127.21.54.130:38415)
I20260811 17:18:19.268558 12251 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.54.130:52669
W20260811 17:18:19.385483 12626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:19.385797 12626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:19.385838 12626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:19.389953 12626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:19.390018 12626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:19.390075 12626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.54.131
I20260811 17:18:19.394906 12626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:35407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.54.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.54.131
--webserver_port=0
--tserver_master_addrs=127.21.54.190:39683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.54.131
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:19.396025 12626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:19.397437 12626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:19.407747 12635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:19.407699 12632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:19.407703 12633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:19.408221 12626 server_base.cc:1034] running on GCE node
I20260811 17:18:19.408761 12626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:19.409497 12626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:19.410738 12626 hybrid_clock.cc:648] HybridClock initialized: now 1786468699410700 us; error 52 us; skew 500 ppm
I20260811 17:18:19.413165 12626 webserver.cc:460] Webserver started at http://127.21.54.131:43633/ using document root <none> and password file <none>
I20260811 17:18:19.413895 12626 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:19.414008 12626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:19.414278 12626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:19.416663 12626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-2/data/instance:
uuid: "54954e8de31744709e849884a46df12a"
format_stamp: "Formatted at 2026-08-11 17:18:19 on dist-test-slave-fd7l"
server_key: "3120761fde8a356da6258a4c3fe2c5fa"
server_key_iv: "6a77f9321e55face114d51aa3a1978c6"
server_key_version: "encryptionkey@0"
I20260811 17:18:19.417296 12626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance:
uuid: "54954e8de31744709e849884a46df12a"
format_stamp: "Formatted at 2026-08-11 17:18:19 on dist-test-slave-fd7l"
server_key: "3120761fde8a356da6258a4c3fe2c5fa"
server_key_iv: "6a77f9321e55face114d51aa3a1978c6"
server_key_version: "encryptionkey@0"
I20260811 17:18:19.421422 12626 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:18:19.424177 12642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:19.425550 12626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:19.425699 12626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-2/wal
uuid: "54954e8de31744709e849884a46df12a"
format_stamp: "Formatted at 2026-08-11 17:18:19 on dist-test-slave-fd7l"
server_key: "3120761fde8a356da6258a4c3fe2c5fa"
server_key_iv: "6a77f9321e55face114d51aa3a1978c6"
server_key_version: "encryptionkey@0"
I20260811 17:18:19.425823 12626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:19.445235 12626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:19.446022 12626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:19.446254 12626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:19.446933 12626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:18:19.447965 12626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:18:19.448041 12626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:19.448117 12626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:18:19.448163 12626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:19.458094 12626 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.131:36721
I20260811 17:18:19.458117 12773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.131:36721 every 8 connection(s)
I20260811 17:18:19.459092 12626 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-2/data/info.pb
I20260811 17:18:19.464773 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 12626
I20260811 17:18:19.464913 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-2/wal/instance
I20260811 17:18:19.465220 21722 external_mini_cluster.cc:1428] Setting key 1b0a5c35f4a01f478c0fa06615c8efd0
I20260811 17:18:19.470177 12296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:19.461177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.131:38531 (local address 127.21.54.190:39683)
0811 17:18:19.461330 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:19.461333 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:19.462450 (+ 1117us) server_negotiation.cc:407] Connection header received
0811 17:18:19.463895 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:19.463898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:19.463959 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:19.464051 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:19.465256 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.466243 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.467794 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.468151 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.468219 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:19.469630 (+ 1411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:19.469636 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:19.469638 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:19.469656 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:19.469679 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:19.469688 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:19.469761 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:19.469944 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 17:18:19.470011 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:18:19.471352 12777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:19.461562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:39683 (local address 127.21.54.131:38531)
0811 17:18:19.462246 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:19.462294 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:19.463671 (+ 1377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:19.464235 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:19.464242 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:19.464323 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:19.465038 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:19.465074 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.466422 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.466426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:19.467648 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:19.467659 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.468381 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.468383 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:19.468586 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:19.469333 (+ 747us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:18:19.469335 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:19.469378 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:18:19.469445 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:19.469781 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:19.469790 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:19.469796 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 17:18:19.469944 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 17:18:19.470245 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":126,"threads_started":1}
I20260811 17:18:19.473006 12774 heartbeater.cc:347] Connected to a master server at 127.21.54.190:39683
I20260811 17:18:19.473281 12774 heartbeater.cc:464] Registering TS with master...
I20260811 17:18:19.473884 12774 heartbeater.cc:511] Master 127.21.54.190:39683 requested a full tablet report, sending...
I20260811 17:18:19.475024 12251 ts_manager.cc:194] Registered new tserver with Master: 54954e8de31744709e849884a46df12a (127.21.54.131:36721)
I20260811 17:18:19.475732 12251 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.54.131:38531
I20260811 17:18:19.479887 21722 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:18:19.486032 12296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:19.481963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42078 (local address 127.21.54.190:39683)
0811 17:18:19.482133 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:19.482150 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:19.482353 (+ 203us) server_negotiation.cc:407] Connection header received
0811 17:18:19.482510 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:19.482513 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:19.482568 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:19.482665 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:19.482966 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.483489 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.484347 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.484629 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.484741 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:19.485570 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:19.485574 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:19.485577 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:19.485590 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:19.485610 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:19.485625 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:19.485705 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:19.485829 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 17:18:19.485886 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:18:19.489167 12251 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:18:19.492120 12251 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:18:19.507055 12789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:19.501511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:36721 (local address 127.0.0.1:37948)
0811 17:18:19.502371 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:19.502386 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:19.502491 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:19.503023 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:19.503030 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:19.503052 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:19.503206 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:19.503213 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.504281 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.504285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:19.505544 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:19.505554 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.506477 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.506480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:19.506602 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:19.506646 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 17:18:19.506767 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 17:18:19.506891 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"thread_start_us":87,"threads_started":1}
I20260811 17:18:19.507144 12793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:19.501948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37948 (local address 127.21.54.131:36721)
0811 17:18:19.502482 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:19.502498 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:19.502528 (+ 30us) server_negotiation.cc:407] Connection header received
0811 17:18:19.502611 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:19.502619 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:19.502841 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:19.503033 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:19.503337 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.504134 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.505672 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.506341 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.506465 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:19.506663 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:19.506815 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 17:18:19.506919 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":147,"threads_started":1}
I20260811 17:18:19.507555 12787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:19.501099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:37793 (local address 127.0.0.1:43186)
0811 17:18:19.501660 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:19.501714 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:19.501844 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:19.503346 (+ 1502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:19.503350 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:19.503370 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:19.503464 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:19.503470 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.504974 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.504977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:19.506074 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:19.506082 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.507138 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.507143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:19.507241 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:19.507267 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 17:18:19.507325 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 17:18:19.507395 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":99,"threads_started":1}
I20260811 17:18:19.507589 12791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:19.501361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:38415 (local address 127.0.0.1:50356)
0811 17:18:19.502682 (+ 1321us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:19.502695 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:19.502790 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:19.503202 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:19.503205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:19.503221 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:19.503313 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:19.503321 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.504772 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.504775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:19.505974 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:19.505985 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.507226 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.507229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:19.507324 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:19.507354 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 17:18:19.507436 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 17:18:19.507493 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1264,"mutex_wait_us":436,"thread_start_us":101,"threads_started":1}
I20260811 17:18:19.507763 12792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:19.501937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50356 (local address 127.21.54.130:38415)
0811 17:18:19.502320 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:19.502333 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:19.502787 (+ 454us) server_negotiation.cc:407] Connection header received
0811 17:18:19.502889 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:19.502893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:19.503073 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:19.503216 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:19.503398 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.504645 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.506133 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.506868 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.506973 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:19.507158 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:19.507487 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 17:18:19.507591 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":141,"threads_started":1}
I20260811 17:18:19.509400 12790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:19.501483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43186 (local address 127.21.54.129:37793)
0811 17:18:19.502773 (+ 1290us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:19.502784 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:19.502806 (+ 22us) server_negotiation.cc:407] Connection header received
0811 17:18:19.502885 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:19.502890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:19.503067 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:19.503625 (+ 558us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:19.503764 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.504855 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.506209 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.506989 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.509031 (+ 2042us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:19.509169 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:19.509208 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 17:18:19.509277 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1175,"thread_start_us":149,"threads_started":1}
I20260811 17:18:19.510977 12699 tablet_service.cc:1467] Processing CreateTablet for tablet 43e3949dd0744e0a8786027dbf3d6006 (DEFAULT_TABLE table=test-table [id=cbdbc68896914c70b0623e78605a7010]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:18:19.510975 12547 tablet_service.cc:1467] Processing CreateTablet for tablet 43e3949dd0744e0a8786027dbf3d6006 (DEFAULT_TABLE table=test-table [id=cbdbc68896914c70b0623e78605a7010]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:18:19.511656 12393 tablet_service.cc:1467] Processing CreateTablet for tablet 43e3949dd0744e0a8786027dbf3d6006 (DEFAULT_TABLE table=test-table [id=cbdbc68896914c70b0623e78605a7010]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:18:19.512151 12547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43e3949dd0744e0a8786027dbf3d6006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:19.512151 12699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43e3949dd0744e0a8786027dbf3d6006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:19.512689 12393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43e3949dd0744e0a8786027dbf3d6006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:19.519429 12794 tablet_bootstrap.cc:492] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c: Bootstrap starting.
I20260811 17:18:19.520275 12796 tablet_bootstrap.cc:492] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a: Bootstrap starting.
I20260811 17:18:19.520275 12795 tablet_bootstrap.cc:492] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4: Bootstrap starting.
I20260811 17:18:19.521756 12794 tablet_bootstrap.cc:654] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:19.522547 12796 tablet_bootstrap.cc:654] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:19.522547 12795 tablet_bootstrap.cc:654] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:19.522820 12794 log.cc:824] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:19.523432 12796 log.cc:824] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:19.523430 12795 log.cc:824] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:19.525193 12794 tablet_bootstrap.cc:492] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c: No bootstrap required, opened a new log
I20260811 17:18:19.525403 12794 ts_tablet_manager.cc:1397] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 17:18:19.525724 12796 tablet_bootstrap.cc:492] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a: No bootstrap required, opened a new log
I20260811 17:18:19.525892 12795 tablet_bootstrap.cc:492] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4: No bootstrap required, opened a new log
I20260811 17:18:19.525923 12796 ts_tablet_manager.cc:1397] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 17:18:19.526090 12795 ts_tablet_manager.cc:1397] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 17:18:19.529675 12796 raft_consensus.cc:348] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1d8776265c4f92b24c1dde6489b69c" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37793 } } peers { permanent_uuid: "5e403d38fad447389a39eeb605993ad4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38415 } } peers { permanent_uuid: "54954e8de31744709e849884a46df12a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36721 } }
I20260811 17:18:19.530035 12796 raft_consensus.cc:374] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:19.530102 12796 raft_consensus.cc:729] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54954e8de31744709e849884a46df12a, State: Initialized, Role: FOLLOWER
I20260811 17:18:19.530124 12794 raft_consensus.cc:348] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1d8776265c4f92b24c1dde6489b69c" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37793 } } peers { permanent_uuid: "5e403d38fad447389a39eeb605993ad4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38415 } } peers { permanent_uuid: "54954e8de31744709e849884a46df12a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36721 } }
I20260811 17:18:19.530377 12794 raft_consensus.cc:374] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:19.530431 12794 raft_consensus.cc:729] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e1d8776265c4f92b24c1dde6489b69c, State: Initialized, Role: FOLLOWER
I20260811 17:18:19.530295 12795 raft_consensus.cc:348] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1d8776265c4f92b24c1dde6489b69c" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37793 } } peers { permanent_uuid: "5e403d38fad447389a39eeb605993ad4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38415 } } peers { permanent_uuid: "54954e8de31744709e849884a46df12a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36721 } }
I20260811 17:18:19.530541 12795 raft_consensus.cc:374] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:19.530592 12795 raft_consensus.cc:729] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e403d38fad447389a39eeb605993ad4, State: Initialized, Role: FOLLOWER
I20260811 17:18:19.530601 12796 consensus_queue.cc:260] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1d8776265c4f92b24c1dde6489b69c" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37793 } } peers { permanent_uuid: "5e403d38fad447389a39eeb605993ad4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38415 } } peers { permanent_uuid: "54954e8de31744709e849884a46df12a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36721 } }
I20260811 17:18:19.530861 12794 consensus_queue.cc:260] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1d8776265c4f92b24c1dde6489b69c" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37793 } } peers { permanent_uuid: "5e403d38fad447389a39eeb605993ad4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38415 } } peers { permanent_uuid: "54954e8de31744709e849884a46df12a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36721 } }
I20260811 17:18:19.530977 12795 consensus_queue.cc:260] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1d8776265c4f92b24c1dde6489b69c" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37793 } } peers { permanent_uuid: "5e403d38fad447389a39eeb605993ad4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38415 } } peers { permanent_uuid: "54954e8de31744709e849884a46df12a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36721 } }
I20260811 17:18:19.531927 12794 ts_tablet_manager.cc:1428] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260811 17:18:19.531919 12795 ts_tablet_manager.cc:1428] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 17:18:19.531939 12796 ts_tablet_manager.cc:1428] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260811 17:18:19.532317 12774 heartbeater.cc:503] Master 127.21.54.190:39683 was elected leader, sending a full tablet report...
I20260811 17:18:19.532678 12623 heartbeater.cc:503] Master 127.21.54.190:39683 was elected leader, sending a full tablet report...
I20260811 17:18:19.532680 12471 heartbeater.cc:503] Master 127.21.54.190:39683 was elected leader, sending a full tablet report...
W20260811 17:18:19.552495 12472 tablet.cc:2369] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:18:19.582302 12802 raft_consensus.cc:482] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:18:19.582584 12802 raft_consensus.cc:504] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1d8776265c4f92b24c1dde6489b69c" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37793 } } peers { permanent_uuid: "5e403d38fad447389a39eeb605993ad4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38415 } } peers { permanent_uuid: "54954e8de31744709e849884a46df12a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36721 } }
I20260811 17:18:19.583824 12802 leader_election.cc:288] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e403d38fad447389a39eeb605993ad4 (127.21.54.130:38415), 54954e8de31744709e849884a46df12a (127.21.54.131:36721)
I20260811 17:18:19.587744 12792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:19.584183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:35419 (local address 127.21.54.130:38415)
0811 17:18:19.584440 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:19.584445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:19.584473 (+ 28us) server_negotiation.cc:407] Connection header received
0811 17:18:19.584704 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:19.584712 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:19.584757 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:19.584843 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:19.585153 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.585667 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.586758 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.587154 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.587285 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:19.587341 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:19.587524 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 17:18:19.587599 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260811 17:18:19.587772 12804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:19.584035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.130:38415 (local address 127.21.54.129:35419)
0811 17:18:19.584396 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:19.584413 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:19.584554 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:19.584859 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:19.584862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:19.584879 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:19.585033 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:19.585038 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.585782 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.585786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:19.586622 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:19.586631 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.587281 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.587284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:19.587383 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:19.587412 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 17:18:19.587505 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 17:18:19.587579 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":111,"threads_started":1}
I20260811 17:18:19.587762 12793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:19.584310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.54.129:60355 (local address 127.21.54.131:36721)
0811 17:18:19.584612 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:19.584618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:19.584656 (+ 38us) server_negotiation.cc:407] Connection header received
0811 17:18:19.584829 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:19.584837 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:19.584877 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:19.584973 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:18:19.585258 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.585746 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.586802 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.587187 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.587317 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:19.587368 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:19.587565 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 17:18:19.587634 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217}
I20260811 17:18:19.587791 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:19.584174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.131:36721 (local address 127.21.54.129:60355)
0811 17:18:19.584574 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:19.584588 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:19.584706 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:19.584984 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:19.584987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:19.585004 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:18:19.585140 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:19.585145 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.585862 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.585867 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:19.586667 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:19.586676 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.587313 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.587317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:19.587415 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:19.587447 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 17:18:19.587534 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 17:18:19.587618 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":1,"thread_start_us":144,"threads_started":1}
I20260811 17:18:19.588765 12723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e3949dd0744e0a8786027dbf3d6006" candidate_uuid: "9e1d8776265c4f92b24c1dde6489b69c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54954e8de31744709e849884a46df12a" is_pre_election: true
I20260811 17:18:19.588755 12571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e3949dd0744e0a8786027dbf3d6006" candidate_uuid: "9e1d8776265c4f92b24c1dde6489b69c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e403d38fad447389a39eeb605993ad4" is_pre_election: true
I20260811 17:18:19.589085 12723 raft_consensus.cc:2454] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e1d8776265c4f92b24c1dde6489b69c in term 0.
I20260811 17:18:19.589087 12571 raft_consensus.cc:2454] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e1d8776265c4f92b24c1dde6489b69c in term 0.
I20260811 17:18:19.589596 12339 leader_election.cc:302] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54954e8de31744709e849884a46df12a, 9e1d8776265c4f92b24c1dde6489b69c; no voters:
I20260811 17:18:19.589869 12802 raft_consensus.cc:2781] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:18:19.589968 12802 raft_consensus.cc:482] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:18:19.590015 12802 raft_consensus.cc:3037] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:19.591074 12802 raft_consensus.cc:504] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1d8776265c4f92b24c1dde6489b69c" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37793 } } peers { permanent_uuid: "5e403d38fad447389a39eeb605993ad4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38415 } } peers { permanent_uuid: "54954e8de31744709e849884a46df12a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36721 } }
I20260811 17:18:19.591424 12802 leader_election.cc:288] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c [CANDIDATE]: Term 1 election: Requested vote from peers 5e403d38fad447389a39eeb605993ad4 (127.21.54.130:38415), 54954e8de31744709e849884a46df12a (127.21.54.131:36721)
I20260811 17:18:19.591790 12571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e3949dd0744e0a8786027dbf3d6006" candidate_uuid: "9e1d8776265c4f92b24c1dde6489b69c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e403d38fad447389a39eeb605993ad4"
I20260811 17:18:19.591795 12723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e3949dd0744e0a8786027dbf3d6006" candidate_uuid: "9e1d8776265c4f92b24c1dde6489b69c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54954e8de31744709e849884a46df12a"
I20260811 17:18:19.591898 12571 raft_consensus.cc:3037] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:19.591938 12723 raft_consensus.cc:3037] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:19.593024 12571 raft_consensus.cc:2454] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e1d8776265c4f92b24c1dde6489b69c in term 1.
I20260811 17:18:19.593112 12723 raft_consensus.cc:2454] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e1d8776265c4f92b24c1dde6489b69c in term 1.
I20260811 17:18:19.593361 12341 leader_election.cc:302] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e403d38fad447389a39eeb605993ad4, 9e1d8776265c4f92b24c1dde6489b69c; no voters:
I20260811 17:18:19.593600 12802 raft_consensus.cc:2781] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:19.593851 12802 raft_consensus.cc:686] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c [term 1 LEADER]: Becoming Leader. State: Replica: 9e1d8776265c4f92b24c1dde6489b69c, State: Running, Role: LEADER
I20260811 17:18:19.594211 12802 consensus_queue.cc:237] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1d8776265c4f92b24c1dde6489b69c" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37793 } } peers { permanent_uuid: "5e403d38fad447389a39eeb605993ad4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38415 } } peers { permanent_uuid: "54954e8de31744709e849884a46df12a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36721 } }
I20260811 17:18:19.597645 12249 catalog_manager.cc:5495] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e1d8776265c4f92b24c1dde6489b69c (127.21.54.129). New cstate: current_term: 1 leader_uuid: "9e1d8776265c4f92b24c1dde6489b69c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e1d8776265c4f92b24c1dde6489b69c" member_type: VOTER last_known_addr { host: "127.21.54.129" port: 37793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e403d38fad447389a39eeb605993ad4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54954e8de31744709e849884a46df12a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36721 } health_report { overall_health: UNKNOWN } } }
I20260811 17:18:19.632795 12790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:19.628922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43198 (local address 127.21.54.129:37793)
0811 17:18:19.629091 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:19.629097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:19.629113 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:18:19.629325 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:19.629328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:19.629388 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:19.629543 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:18:19.629824 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.630339 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.631215 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:19.631437 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:19.631553 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:19.631880 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:18:19.632333 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:18:19.632478 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:19.632585 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 17:18:19.632642 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 17:18:19.639050 12571 raft_consensus.cc:1261] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4 [term 1 FOLLOWER]: Refusing update from remote peer 9e1d8776265c4f92b24c1dde6489b69c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:18:19.639050 12723 raft_consensus.cc:1261] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a [term 1 FOLLOWER]: Refusing update from remote peer 9e1d8776265c4f92b24c1dde6489b69c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:18:19.639847 12806 consensus_queue.cc:1035] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c [LEADER]: Connected to new peer: Peer: permanent_uuid: "54954e8de31744709e849884a46df12a" member_type: VOTER last_known_addr { host: "127.21.54.131" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:18:19.640070 12802 consensus_queue.cc:1035] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e403d38fad447389a39eeb605993ad4" member_type: VOTER last_known_addr { host: "127.21.54.130" port: 38415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:18:19.648787 12811 mvcc.cc:205] Tried to move back new op lower bound from 7317375793713315840 to 7317375793540571136. Current Snapshot: MvccSnapshot[applied={T|T < 7317375793713315840}]
I20260811 17:18:19.657194 12249 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:18:19.657401 12249 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:18:19.660017 12249 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6528593d1ea04e83ae4bce8090e98ed1: Sending DeleteTablet for 3 replicas of tablet 43e3949dd0744e0a8786027dbf3d6006
I20260811 17:18:19.660898 12393 tablet_service.cc:1514] Processing DeleteTablet for tablet 43e3949dd0744e0a8786027dbf3d6006 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:18:19 UTC) from {username='slave'} at 127.0.0.1:43186
I20260811 17:18:19.660972 12547 tablet_service.cc:1514] Processing DeleteTablet for tablet 43e3949dd0744e0a8786027dbf3d6006 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:18:19 UTC) from {username='slave'} at 127.0.0.1:50356
I20260811 17:18:19.661064 12699 tablet_service.cc:1514] Processing DeleteTablet for tablet 43e3949dd0744e0a8786027dbf3d6006 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:18:19 UTC) from {username='slave'} at 127.0.0.1:37948
I20260811 17:18:19.661580 12822 tablet_replica.cc:330] T 43e3949dd0744e0a8786027dbf3d6006 P 9e1d8776265c4f92b24c1dde6489b69c: stopping tablet replica
I20260811 17:18:19.661887 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 12322
I20260811 17:18:19.665380 12824 tablet_replica.cc:330] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4: stopping tablet replica
I20260811 17:18:19.665683 12823 tablet_replica.cc:330] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a: stopping tablet replica
I20260811 17:18:19.665762 12824 raft_consensus.cc:2229] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:18:19.665941 12823 raft_consensus.cc:2229] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:18:19.666141 12824 raft_consensus.cc:2258] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:18:19.666214 12823 raft_consensus.cc:2258] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:18:19.667769 12823 ts_tablet_manager.cc:1905] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:18:19.667769 12824 ts_tablet_manager.cc:1905] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:18:19.670683 12824 ts_tablet_manager.cc:1918] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:18:19.670696 12823 ts_tablet_manager.cc:1918] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:18:19.670797 12823 log.cc:1196] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-2/wal/wals/43e3949dd0744e0a8786027dbf3d6006
I20260811 17:18:19.670799 12824 log.cc:1196] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4: Deleting WAL directory at /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468428054096-21722-0/minicluster-data/ts-1/wal/wals/43e3949dd0744e0a8786027dbf3d6006
I20260811 17:18:19.671139 12824 ts_tablet_manager.cc:1939] T 43e3949dd0744e0a8786027dbf3d6006 P 5e403d38fad447389a39eeb605993ad4: Deleting consensus metadata
I20260811 17:18:19.671187 12823 ts_tablet_manager.cc:1939] T 43e3949dd0744e0a8786027dbf3d6006 P 54954e8de31744709e849884a46df12a: Deleting consensus metadata
I20260811 17:18:19.672549 12236 catalog_manager.cc:4841] TS 5e403d38fad447389a39eeb605993ad4 (127.21.54.130:38415): tablet 43e3949dd0744e0a8786027dbf3d6006 (table test-table [id=cbdbc68896914c70b0623e78605a7010]) successfully deleted
I20260811 17:18:19.672688 12234 catalog_manager.cc:4841] TS 54954e8de31744709e849884a46df12a (127.21.54.131:36721): tablet 43e3949dd0744e0a8786027dbf3d6006 (table test-table [id=cbdbc68896914c70b0623e78605a7010]) successfully deleted
W20260811 17:18:19.672747 12235 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.54.129:37793 (error 108)
I20260811 17:18:19.673292 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 12475
I20260811 17:18:19.673938 12791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:19.673537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.129:37793 (local address 127.0.0.1:43208)
0811 17:18:19.673753 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:19.673835 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.129:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260811 17:18:19.675779 12235 catalog_manager.cc:4584] TS 9e1d8776265c4f92b24c1dde6489b69c (127.21.54.129:37793): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 43e3949dd0744e0a8786027dbf3d6006: Network error: Client connection negotiation failed: client connection to 127.21.54.129:37793: connect: Connection refused (error 111)
I20260811 17:18:19.682601 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 12626
I20260811 17:18:19.690636 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 12218
2026-08-11T17:18:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1134 ms)
[----------] 6 tests from EncryptionPolicyTest (11374 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:19 dist-test-slave-fd7l krb5kdc[12829](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:19 dist-test-slave-fd7l krb5kdc[12829](info): set up 2 sockets
Aug 11 17:18:19 dist-test-slave-fd7l krb5kdc[12829](info): commencing operation
krb5kdc: starting...
W20260811 17:18:20.891511 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:20 dist-test-slave-fd7l krb5kdc[12829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:18:21.033244 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46859
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:21.152227 12844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:21.152729 12844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:21.152841 12844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:21.156946 12844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:21.157051 12844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:21.157086 12844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:21.157126 12844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:21.157161 12844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:21.162806 12844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46859
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:21.164026 12844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:21.165170 12844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:21.174548 12852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:21.174705 12849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:21.174996 12850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:21.174986 12844 server_base.cc:1034] running on GCE node
I20260811 17:18:21.175726 12844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:21.177069 12844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:21.178279 12844 hybrid_clock.cc:648] HybridClock initialized: now 1786468701178224 us; error 71 us; skew 500 ppm
Aug 11 17:18:21 dist-test-slave-fd7l krb5kdc[12829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468701, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:21.181635 12844 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:18:21.182679 12844 webserver.cc:460] Webserver started at http://127.0.0.1:40905/ using document root <none> and password file <none>
I20260811 17:18:21.183252 12844 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:21.183317 12844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:21.183511 12844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:21.185802 12844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "8a201ae92d324489ac43ff7f6ce3fa42"
format_stamp: "Formatted at 2026-08-11 17:18:21 on dist-test-slave-fd7l"
server_key: "cb187088f00dbcf4ac2480be93c4dc52"
server_key_iv: "d709d9b05eb5dd3de70fe8e89acfd35a"
server_key_version: "encryptionkey@0"
I20260811 17:18:21.186369 12844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "8a201ae92d324489ac43ff7f6ce3fa42"
format_stamp: "Formatted at 2026-08-11 17:18:21 on dist-test-slave-fd7l"
server_key: "cb187088f00dbcf4ac2480be93c4dc52"
server_key_iv: "d709d9b05eb5dd3de70fe8e89acfd35a"
server_key_version: "encryptionkey@0"
I20260811 17:18:21.190485 12844 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:18:21.193253 12859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:21.194814 12844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:21.195017 12844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "8a201ae92d324489ac43ff7f6ce3fa42"
format_stamp: "Formatted at 2026-08-11 17:18:21 on dist-test-slave-fd7l"
server_key: "cb187088f00dbcf4ac2480be93c4dc52"
server_key_iv: "d709d9b05eb5dd3de70fe8e89acfd35a"
server_key_version: "encryptionkey@0"
I20260811 17:18:21.195154 12844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:21.217336 12844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:21.221402 12844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:21.221662 12844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:21.230032 12844 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46859
I20260811 17:18:21.230114 12916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46859 every 8 connection(s)
I20260811 17:18:21.231187 12844 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:21.231660 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 12844
I20260811 17:18:21.231830 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:21.232170 21722 external_mini_cluster.cc:1428] Setting key e1325aa2da2796de860eaa94b9eef678
I20260811 17:18:21.234889 12918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:21.242357 12918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42: Bootstrap starting.
Aug 11 17:18:21 dist-test-slave-fd7l krb5kdc[12829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 17:18:21.245101 12918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:21.246075 12918 log.cc:824] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:21.248697 12918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42: No bootstrap required, opened a new log
I20260811 17:18:21.250378 12922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:21.233776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42427 (local address 127.0.0.1:46859)
0811 17:18:21.234273 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:21.234293 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:21.234338 (+ 45us) server_negotiation.cc:407] Connection header received
0811 17:18:21.235211 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:21.235235 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:21.235698 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:21.236067 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:21.237399 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:21.238997 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:21.240103 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:21.240862 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:21.241054 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:21.244052 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:21.244059 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:21.244075 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:21.244108 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:21.246640 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:21.247295 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:21.247301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:21.247307 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:21.247443 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:21.247801 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:21.247804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:21.247806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:21.248313 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:21.248555 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:21.248920 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 17:18:21.249172 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":148,"threads_started":1}
I20260811 17:18:21.252668 12918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a201ae92d324489ac43ff7f6ce3fa42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46859 } }
I20260811 17:18:21.252920 12918 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:21.252985 12918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a201ae92d324489ac43ff7f6ce3fa42, State: Initialized, Role: FOLLOWER
I20260811 17:18:21.253495 12918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a201ae92d324489ac43ff7f6ce3fa42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46859 } }
I20260811 17:18:21.253651 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:21.253681 12918 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:21.253746 12918 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:21.253849 12918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:21.254804 12918 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a201ae92d324489ac43ff7f6ce3fa42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46859 } }
I20260811 17:18:21.255172 12918 leader_election.cc:302] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a201ae92d324489ac43ff7f6ce3fa42; no voters:
I20260811 17:18:21.255487 12918 leader_election.cc:288] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:21.255637 12928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:21.255908 12928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42 [term 1 LEADER]: Becoming Leader. State: Replica: 8a201ae92d324489ac43ff7f6ce3fa42, State: Running, Role: LEADER
I20260811 17:18:21.256268 12928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a201ae92d324489ac43ff7f6ce3fa42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46859 } }
I20260811 17:18:21.256894 12918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:21.257635 12932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a201ae92d324489ac43ff7f6ce3fa42. Latest consensus state: current_term: 1 leader_uuid: "8a201ae92d324489ac43ff7f6ce3fa42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a201ae92d324489ac43ff7f6ce3fa42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46859 } } }
I20260811 17:18:21.258500 12932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:21.258350 12931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a201ae92d324489ac43ff7f6ce3fa42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a201ae92d324489ac43ff7f6ce3fa42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46859 } } }
I20260811 17:18:21.258883 12931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42 [sys.catalog]: This master's current role is: LEADER
W20260811 17:18:21.262569 12950 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:18:21.262679 12950 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:18:21.262753 12947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:21.263572 12947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:21.266290 12922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:21.256252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44603 (local address 127.0.0.1:46859)
0811 17:18:21.256358 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:21.256363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:21.256416 (+ 53us) server_negotiation.cc:407] Connection header received
0811 17:18:21.256651 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:21.256655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:21.256703 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:21.256777 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:21.257678 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:21.258417 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:21.259568 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:21.259905 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:21.260006 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:21.261936 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:21.261940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:21.261953 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:21.261985 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:21.263932 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:21.264566 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:21.264571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:21.264573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:21.264676 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:21.265103 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:21.265107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:21.265109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:21.265411 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:21.265579 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:21.266028 (+ 449us) server_negotiation.cc:299] Negotiation successful
0811 17:18:21.266132 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 17:18:21.269595 12947 catalog_manager.cc:1347] Generated new cluster ID: 6287b393006945af887022682613351f
I20260811 17:18:21.269659 12947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:21.281538 12947 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:21.282986 12947 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:21.290372 12947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a201ae92d324489ac43ff7f6ce3fa42: Generated new TSK 0
I20260811 17:18:21.291270 12947 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:21.295662 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 12844
W20260811 17:18:21.304903 12827 connection.cc:535] client connection to 127.0.0.1:46859 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46859): Connection reset by peer (error 104)
2026-08-11T17:18:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1603 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:21 dist-test-slave-fd7l krb5kdc[12956](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:21 dist-test-slave-fd7l krb5kdc[12956](info): set up 2 sockets
Aug 11 17:18:21 dist-test-slave-fd7l krb5kdc[12956](info): commencing operation
krb5kdc: starting...
W20260811 17:18:22.481320 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:22 dist-test-slave-fd7l krb5kdc[12956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:18:22.620038 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36851
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:22.741580 12970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:22.741895 12970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:22.741981 12970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:22.746112 12970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:22.746209 12970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:22.746249 12970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:22.746309 12970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:22.746347 12970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:22.751863 12970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36851
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:22.753062 12970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:22.754132 12970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:22.763754 12979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:22.763775 12977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:22.764125 12970 server_base.cc:1034] running on GCE node
W20260811 17:18:22.763761 12975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:22.764842 12970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:22.765946 12970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:22.767146 12970 hybrid_clock.cc:648] HybridClock initialized: now 1786468702767118 us; error 48 us; skew 500 ppm
Aug 11 17:18:22 dist-test-slave-fd7l krb5kdc[12956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468702, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:22.770159 12970 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:18:22.771130 12970 webserver.cc:460] Webserver started at http://127.0.0.1:44897/ using document root <none> and password file <none>
I20260811 17:18:22.771670 12970 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:22.771729 12970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:22.771899 12970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:22.774087 12970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "dad3c959febd4dc6a429c78881b4dd16"
format_stamp: "Formatted at 2026-08-11 17:18:22 on dist-test-slave-fd7l"
server_key: "99714787af88d27be130ce8f4ab5eee9"
server_key_iv: "4b108204c5a88ae8ca075f9342c1ad10"
server_key_version: "encryptionkey@0"
I20260811 17:18:22.774626 12970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "dad3c959febd4dc6a429c78881b4dd16"
format_stamp: "Formatted at 2026-08-11 17:18:22 on dist-test-slave-fd7l"
server_key: "99714787af88d27be130ce8f4ab5eee9"
server_key_iv: "4b108204c5a88ae8ca075f9342c1ad10"
server_key_version: "encryptionkey@0"
I20260811 17:18:22.778589 12970 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:18:22.781342 12986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:22.782771 12970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:22.782927 12970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "dad3c959febd4dc6a429c78881b4dd16"
format_stamp: "Formatted at 2026-08-11 17:18:22 on dist-test-slave-fd7l"
server_key: "99714787af88d27be130ce8f4ab5eee9"
server_key_iv: "4b108204c5a88ae8ca075f9342c1ad10"
server_key_version: "encryptionkey@0"
I20260811 17:18:22.783070 12970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:22.802659 12970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:22.806594 12970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:22.806842 12970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:22.815312 12970 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36851
I20260811 17:18:22.815340 13047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36851 every 8 connection(s)
I20260811 17:18:22.816430 12970 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:22.818436 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 12970
I20260811 17:18:22.818614 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:22.818955 21722 external_mini_cluster.cc:1428] Setting key b35b6dad85a2f851cb1ae4a5609fc4c3
I20260811 17:18:22.819937 13048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:22.826573 13048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16: Bootstrap starting.
I20260811 17:18:22.829212 13051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:22.820655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47247 (local address 127.0.0.1:36851)
0811 17:18:22.821171 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:22.821187 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:22.821226 (+ 39us) server_negotiation.cc:407] Connection header received
0811 17:18:22.821934 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:22.821952 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:22.822328 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:22.822696 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:22.823116 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:22.824114 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:22.825144 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:22.825629 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:22.825813 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:22.826898 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:22.826905 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:22.826922 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:22.827450 (+ 528us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:22.827539 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:22.827574 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:22.827727 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:22.827970 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 17:18:22.828137 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":170,"threads_started":1}
I20260811 17:18:22.829726 13048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:22.830696 13048 log.cc:824] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:22.832598 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:22.833037 13048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16: No bootstrap required, opened a new log
I20260811 17:18:22.837349 13048 raft_consensus.cc:348] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad3c959febd4dc6a429c78881b4dd16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36851 } }
I20260811 17:18:22.837647 13048 raft_consensus.cc:374] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:22.837714 13048 raft_consensus.cc:729] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad3c959febd4dc6a429c78881b4dd16, State: Initialized, Role: FOLLOWER
I20260811 17:18:22.838212 13048 consensus_queue.cc:260] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad3c959febd4dc6a429c78881b4dd16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36851 } }
I20260811 17:18:22.838349 13048 raft_consensus.cc:388] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:22.838445 13048 raft_consensus.cc:482] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:22.838552 13048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:22.839517 13051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:22.834993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40633 (local address 127.0.0.1:36851)
0811 17:18:22.835145 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:22.835150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:22.835310 (+ 160us) server_negotiation.cc:407] Connection header received
0811 17:18:22.835408 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:22.835411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:22.835457 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:22.835550 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:22.835821 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:22.836631 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:22.837503 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:22.837860 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:22.837972 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:22.838982 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:22.838986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:22.838989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:22.839002 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:22.839029 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:22.839042 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:22.839134 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:22.839328 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 17:18:22.839377 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 17:18:22.839903 13048 raft_consensus.cc:504] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad3c959febd4dc6a429c78881b4dd16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36851 } }
I20260811 17:18:22.840301 13048 leader_election.cc:302] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dad3c959febd4dc6a429c78881b4dd16; no voters:
I20260811 17:18:22.840698 13048 leader_election.cc:288] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:22.840790 13059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:22.841073 13059 raft_consensus.cc:686] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16 [term 1 LEADER]: Becoming Leader. State: Replica: dad3c959febd4dc6a429c78881b4dd16, State: Running, Role: LEADER
I20260811 17:18:22.841389 13059 consensus_queue.cc:237] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad3c959febd4dc6a429c78881b4dd16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36851 } }
I20260811 17:18:22.841985 13048 sys_catalog.cc:564] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:22.843187 13061 sys_catalog.cc:455] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dad3c959febd4dc6a429c78881b4dd16. Latest consensus state: current_term: 1 leader_uuid: "dad3c959febd4dc6a429c78881b4dd16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad3c959febd4dc6a429c78881b4dd16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36851 } } }
I20260811 17:18:22.843345 13061 sys_catalog.cc:458] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:22.843652 13060 sys_catalog.cc:455] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dad3c959febd4dc6a429c78881b4dd16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad3c959febd4dc6a429c78881b4dd16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36851 } } }
I20260811 17:18:22.843765 13060 sys_catalog.cc:458] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:22.843737 13068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:22.847107 13068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:22.853391 13068 catalog_manager.cc:1347] Generated new cluster ID: 22fa3deb51de45b7a68e028531adf3a1
I20260811 17:18:22.853479 13068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:22.871796 13068 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:22.873246 13068 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:22.882957 13068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dad3c959febd4dc6a429c78881b4dd16: Generated new TSK 0
I20260811 17:18:22.883998 13068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:22.888374 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 12970
W20260811 17:18:22.901319 12954 connection.cc:535] client connection to 127.0.0.1:36851 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36851): Connection reset by peer (error 104)
2026-08-11T17:18:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1596 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:22 dist-test-slave-fd7l krb5kdc[13083](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:22 dist-test-slave-fd7l krb5kdc[13083](info): set up 2 sockets
Aug 11 17:18:22 dist-test-slave-fd7l krb5kdc[13083](info): commencing operation
krb5kdc: starting...
W20260811 17:18:24.072111 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:24 dist-test-slave-fd7l krb5kdc[13083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:18:24.212062 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46171
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:24.329882 13099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:24.330183 13099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:24.330228 13099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:24.334373 13099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:24.334450 13099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:24.334470 13099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:24.334492 13099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:24.334507 13099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:24.340020 13099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46171
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:24.341207 13099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:24.342284 13099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:24.351455 13105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:24.351519 13099 server_base.cc:1034] running on GCE node
W20260811 17:18:24.351428 13109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:24.351516 13106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:24.352321 13099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:24.353614 13099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:24.354823 13099 hybrid_clock.cc:648] HybridClock initialized: now 1786468704354808 us; error 30 us; skew 500 ppm
Aug 11 17:18:24 dist-test-slave-fd7l krb5kdc[13083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468704, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:24.364300 13099 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:18:24.365423 13099 webserver.cc:460] Webserver started at http://127.0.0.1:44001/ using document root <none> and password file <none>
I20260811 17:18:24.366056 13099 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:24.366166 13099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:24.366422 13099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:24.368780 13099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "e634c48699c44e90a7a5e8305a793f43"
format_stamp: "Formatted at 2026-08-11 17:18:24 on dist-test-slave-fd7l"
server_key: "04757f876feaa1ead95ed1a3f24f7ecb"
server_key_iv: "61b687edad41643f132f9fc3bbc144aa"
server_key_version: "encryptionkey@0"
I20260811 17:18:24.369472 13099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "e634c48699c44e90a7a5e8305a793f43"
format_stamp: "Formatted at 2026-08-11 17:18:24 on dist-test-slave-fd7l"
server_key: "04757f876feaa1ead95ed1a3f24f7ecb"
server_key_iv: "61b687edad41643f132f9fc3bbc144aa"
server_key_version: "encryptionkey@0"
I20260811 17:18:24.373525 13099 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:18:24.376168 13120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:24.377599 13099 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:18:24.377765 13099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "e634c48699c44e90a7a5e8305a793f43"
format_stamp: "Formatted at 2026-08-11 17:18:24 on dist-test-slave-fd7l"
server_key: "04757f876feaa1ead95ed1a3f24f7ecb"
server_key_iv: "61b687edad41643f132f9fc3bbc144aa"
server_key_version: "encryptionkey@0"
I20260811 17:18:24.377890 13099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:24.382721 13099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:24.386029 13099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:24.386236 13099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:24.394011 13099 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46171
I20260811 17:18:24.394054 13177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46171 every 8 connection(s)
I20260811 17:18:24.395032 13099 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:24.398309 13178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:24.400051 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 13099
I20260811 17:18:24.400250 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:24.400628 21722 external_mini_cluster.cc:1428] Setting key 2e5f55ad45c08bc0f374fb89d86554e1
I20260811 17:18:24.405668 13178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43: Bootstrap starting.
I20260811 17:18:24.406670 13181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:24.402380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56671 (local address 127.0.0.1:46171)
0811 17:18:24.402912 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:24.402929 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:24.402971 (+ 42us) server_negotiation.cc:407] Connection header received
0811 17:18:24.403687 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:24.403705 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:24.404009 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:24.404411 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:24.405356 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:24.405361 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:24.405373 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:24.405384 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:24.405445 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:24.405466 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:24.405635 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:24.405713 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 17:18:24.405892 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":156,"threads_started":1}
I20260811 17:18:24.408186 13178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:24.409139 13178 log.cc:824] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:24.409613 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:24.411706 13178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43: No bootstrap required, opened a new log
I20260811 17:18:24.413915 13181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:24.412059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46721 (local address 127.0.0.1:46171)
0811 17:18:24.412207 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:24.412213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:24.412441 (+ 228us) server_negotiation.cc:407] Connection header received
0811 17:18:24.412600 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:24.412602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:24.412662 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:24.412730 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:24.413519 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:24.413522 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:24.413524 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:24.413535 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:24.413561 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:24.413571 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:24.413669 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:24.413740 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 17:18:24.413773 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 17:18:24.416576 13178 raft_consensus.cc:348] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e634c48699c44e90a7a5e8305a793f43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46171 } }
I20260811 17:18:24.416923 13178 raft_consensus.cc:374] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:24.416998 13178 raft_consensus.cc:729] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e634c48699c44e90a7a5e8305a793f43, State: Initialized, Role: FOLLOWER
I20260811 17:18:24.417479 13178 consensus_queue.cc:260] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e634c48699c44e90a7a5e8305a793f43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46171 } }
I20260811 17:18:24.417640 13178 raft_consensus.cc:388] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:24.417708 13178 raft_consensus.cc:482] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:24.417809 13178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:24.418795 13178 raft_consensus.cc:504] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e634c48699c44e90a7a5e8305a793f43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46171 } }
I20260811 17:18:24.419154 13178 leader_election.cc:302] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e634c48699c44e90a7a5e8305a793f43; no voters:
I20260811 17:18:24.419484 13178 leader_election.cc:288] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:24.419615 13188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:24.419889 13188 raft_consensus.cc:686] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43 [term 1 LEADER]: Becoming Leader. State: Replica: e634c48699c44e90a7a5e8305a793f43, State: Running, Role: LEADER
I20260811 17:18:24.420266 13188 consensus_queue.cc:237] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e634c48699c44e90a7a5e8305a793f43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46171 } }
I20260811 17:18:24.420790 13178 sys_catalog.cc:564] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:24.422129 13190 sys_catalog.cc:455] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e634c48699c44e90a7a5e8305a793f43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e634c48699c44e90a7a5e8305a793f43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46171 } } }
I20260811 17:18:24.422088 13192 sys_catalog.cc:455] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e634c48699c44e90a7a5e8305a793f43. Latest consensus state: current_term: 1 leader_uuid: "e634c48699c44e90a7a5e8305a793f43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e634c48699c44e90a7a5e8305a793f43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46171 } } }
I20260811 17:18:24.422317 13192 sys_catalog.cc:458] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:24.422317 13190 sys_catalog.cc:458] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43 [sys.catalog]: This master's current role is: LEADER
W20260811 17:18:24.426003 13207 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:18:24.426074 13207 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:18:24.426162 13203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:24.426844 13203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:24.432689 13203 catalog_manager.cc:1347] Generated new cluster ID: 18cd11f866c244ccb261d940019284a7
I20260811 17:18:24.432790 13203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:24.453162 13203 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:24.453575 13203 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:24.466439 13203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e634c48699c44e90a7a5e8305a793f43: Generated new TSK 0
I20260811 17:18:24.467314 13203 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:24.478956 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 13099
2026-08-11T17:18:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1585 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:24 dist-test-slave-fd7l krb5kdc[13213](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:24 dist-test-slave-fd7l krb5kdc[13213](info): set up 2 sockets
Aug 11 17:18:24 dist-test-slave-fd7l krb5kdc[13213](info): commencing operation
krb5kdc: starting...
W20260811 17:18:25.629206 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:25 dist-test-slave-fd7l krb5kdc[13213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:18:25.768224 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37103
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:25.886534 13228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:25.886880 13228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:25.886966 13228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:25.891095 13228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:25.891315 13228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:25.891417 13228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:25.891461 13228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:25.891498 13228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:25.897168 13228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:25.898346 13228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:25.899418 13228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:25.908514 13235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:25.908577 13237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:25.908567 13233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:25.908766 13228 server_base.cc:1034] running on GCE node
I20260811 17:18:25.909554 13228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:25.910782 13228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:25.911993 13228 hybrid_clock.cc:648] HybridClock initialized: now 1786468705911977 us; error 30 us; skew 500 ppm
Aug 11 17:18:25 dist-test-slave-fd7l krb5kdc[13213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468705, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:25.914994 13228 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:18:25.916010 13228 webserver.cc:460] Webserver started at http://127.0.0.1:43877/ using document root <none> and password file <none>
I20260811 17:18:25.916706 13228 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:25.916808 13228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:25.917057 13228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:25.919297 13228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "af54f45e2c884c17b0eb02130d5bdaf4"
format_stamp: "Formatted at 2026-08-11 17:18:25 on dist-test-slave-fd7l"
server_key: "cea31427d2c95c25f41ea1ed671cbb78"
server_key_iv: "36449258a3128b29cd702f688cb71d92"
server_key_version: "encryptionkey@0"
I20260811 17:18:25.919899 13228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "af54f45e2c884c17b0eb02130d5bdaf4"
format_stamp: "Formatted at 2026-08-11 17:18:25 on dist-test-slave-fd7l"
server_key: "cea31427d2c95c25f41ea1ed671cbb78"
server_key_iv: "36449258a3128b29cd702f688cb71d92"
server_key_version: "encryptionkey@0"
I20260811 17:18:25.923820 13228 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:18:25.926487 13246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:25.927799 13228 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 17:18:25.927963 13228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "af54f45e2c884c17b0eb02130d5bdaf4"
format_stamp: "Formatted at 2026-08-11 17:18:25 on dist-test-slave-fd7l"
server_key: "cea31427d2c95c25f41ea1ed671cbb78"
server_key_iv: "36449258a3128b29cd702f688cb71d92"
server_key_version: "encryptionkey@0"
I20260811 17:18:25.928084 13228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:25.943382 13228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:25.946713 13228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:25.946955 13228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:25.954716 13228 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37103
I20260811 17:18:25.954746 13309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37103 every 8 connection(s)
I20260811 17:18:25.955746 13228 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:25.956182 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 13228
I20260811 17:18:25.956350 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:25.956739 21722 external_mini_cluster.cc:1428] Setting key e4893e0df8e3760fde348bc74d369152
I20260811 17:18:25.959188 13310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:25.966233 13310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4: Bootstrap starting.
Aug 11 17:18:25 dist-test-slave-fd7l krb5kdc[13213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 17:18:25.969110 13310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:25.970077 13310 log.cc:824] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:25.972316 13310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4: No bootstrap required, opened a new log
I20260811 17:18:25.973078 13313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:25.958249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40043 (local address 127.0.0.1:37103)
0811 17:18:25.958699 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:25.958719 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:25.958764 (+ 45us) server_negotiation.cc:407] Connection header received
0811 17:18:25.959627 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:25.959652 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:25.960086 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:25.960478 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:25.961388 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:25.962973 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:25.963934 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:25.964727 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:25.964909 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:25.967623 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:25.967631 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:25.967649 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:25.967687 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:25.969874 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:25.970432 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:25.970439 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:25.970447 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:25.970611 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:25.970938 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:25.970943 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:25.970945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:25.971375 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:25.971594 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:25.971922 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 17:18:25.972229 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":117,"threads_started":1}
I20260811 17:18:25.976012 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:25.976321 13310 raft_consensus.cc:348] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af54f45e2c884c17b0eb02130d5bdaf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37103 } }
I20260811 17:18:25.976612 13310 raft_consensus.cc:374] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:25.976709 13310 raft_consensus.cc:729] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af54f45e2c884c17b0eb02130d5bdaf4, State: Initialized, Role: FOLLOWER
I20260811 17:18:25.977185 13310 consensus_queue.cc:260] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af54f45e2c884c17b0eb02130d5bdaf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37103 } }
I20260811 17:18:25.977346 13310 raft_consensus.cc:388] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:25.977420 13310 raft_consensus.cc:482] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:25.977535 13310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:25.978734 13310 raft_consensus.cc:504] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af54f45e2c884c17b0eb02130d5bdaf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37103 } }
I20260811 17:18:25.979158 13310 leader_election.cc:302] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af54f45e2c884c17b0eb02130d5bdaf4; no voters:
I20260811 17:18:25.979552 13310 leader_election.cc:288] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:25.979664 13321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:25.979923 13321 raft_consensus.cc:686] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4 [term 1 LEADER]: Becoming Leader. State: Replica: af54f45e2c884c17b0eb02130d5bdaf4, State: Running, Role: LEADER
I20260811 17:18:25.980307 13321 consensus_queue.cc:237] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af54f45e2c884c17b0eb02130d5bdaf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37103 } }
I20260811 17:18:25.980955 13310 sys_catalog.cc:564] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:25.982105 13322 sys_catalog.cc:455] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af54f45e2c884c17b0eb02130d5bdaf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af54f45e2c884c17b0eb02130d5bdaf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37103 } } }
I20260811 17:18:25.982268 13322 sys_catalog.cc:458] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:25.982695 13323 sys_catalog.cc:455] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af54f45e2c884c17b0eb02130d5bdaf4. Latest consensus state: current_term: 1 leader_uuid: "af54f45e2c884c17b0eb02130d5bdaf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af54f45e2c884c17b0eb02130d5bdaf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37103 } } }
I20260811 17:18:25.982899 13323 sys_catalog.cc:458] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:25.983637 13332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:25.987180 13332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:25.987974 13313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:25.978342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34605 (local address 127.0.0.1:37103)
0811 17:18:25.978458 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:25.978462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:25.978687 (+ 225us) server_negotiation.cc:407] Connection header received
0811 17:18:25.978827 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:25.978829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:25.978902 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:25.978975 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:25.979860 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:25.980633 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:25.981824 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:25.982363 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:25.982487 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:25.984651 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:25.984654 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:25.984656 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:25.984682 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:25.986188 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:25.986629 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:25.986635 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:25.986637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:25.986730 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:25.987017 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:25.987021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:25.987022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:25.987218 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:25.987362 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:25.987672 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 17:18:25.987795 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 17:18:25.993075 13332 catalog_manager.cc:1347] Generated new cluster ID: fb0471f62635434aa9fdd4635cdaeecc
I20260811 17:18:25.993146 13332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:26.011150 13332 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:26.012210 13332 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:26.019848 13332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af54f45e2c884c17b0eb02130d5bdaf4: Generated new TSK 0
I20260811 17:18:26.020706 13332 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:26.029309 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 13228
2026-08-11T17:18:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1552 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:26 dist-test-slave-fd7l krb5kdc[13347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:26 dist-test-slave-fd7l krb5kdc[13347](info): set up 2 sockets
Aug 11 17:18:26 dist-test-slave-fd7l krb5kdc[13347](info): commencing operation
krb5kdc: starting...
W20260811 17:18:27.177248 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:27 dist-test-slave-fd7l krb5kdc[13347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:18:27.320245 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38617
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:27.437959 13360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:27.438301 13360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:27.438380 13360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:27.442507 13360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:27.442626 13360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:27.442662 13360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:27.442705 13360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:27.442739 13360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:27.448426 13360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:27.449673 13360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:27.450757 13360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:27.459851 13365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:27.459898 13360 server_base.cc:1034] running on GCE node
W20260811 17:18:27.459844 13366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:27.459878 13368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:27.460810 13360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:27.461959 13360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:27.463195 13360 hybrid_clock.cc:648] HybridClock initialized: now 1786468707463155 us; error 50 us; skew 500 ppm
Aug 11 17:18:27 dist-test-slave-fd7l krb5kdc[13347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468707, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:27.467311 13360 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:18:27.468571 13360 webserver.cc:460] Webserver started at http://127.0.0.1:35209/ using document root <none> and password file <none>
I20260811 17:18:27.469280 13360 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:27.469451 13360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:27.469691 13360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:27.472626 13360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "3e5c8833c3384d57a55f08a24dd461b3"
format_stamp: "Formatted at 2026-08-11 17:18:27 on dist-test-slave-fd7l"
server_key: "85d710ce46740d1a157050fb50bb4ac8"
server_key_iv: "185bc6b810a6dbe08add019be218de4b"
server_key_version: "encryptionkey@0"
I20260811 17:18:27.473424 13360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "3e5c8833c3384d57a55f08a24dd461b3"
format_stamp: "Formatted at 2026-08-11 17:18:27 on dist-test-slave-fd7l"
server_key: "85d710ce46740d1a157050fb50bb4ac8"
server_key_iv: "185bc6b810a6dbe08add019be218de4b"
server_key_version: "encryptionkey@0"
I20260811 17:18:27.478350 13360 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 17:18:27.481143 13376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:27.482689 13360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:27.482868 13360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "3e5c8833c3384d57a55f08a24dd461b3"
format_stamp: "Formatted at 2026-08-11 17:18:27 on dist-test-slave-fd7l"
server_key: "85d710ce46740d1a157050fb50bb4ac8"
server_key_iv: "185bc6b810a6dbe08add019be218de4b"
server_key_version: "encryptionkey@0"
I20260811 17:18:27.483024 13360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:27.511566 13360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:27.514998 13360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:27.515264 13360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:27.523069 13360 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38617
I20260811 17:18:27.523094 13437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38617 every 8 connection(s)
I20260811 17:18:27.524153 13360 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:27.527431 13438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:27.530769 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 13360
I20260811 17:18:27.530957 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:27.531330 21722 external_mini_cluster.cc:1428] Setting key affd3ae46c5e27303f5a7ad17a9160e2
I20260811 17:18:27.535001 13438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3: Bootstrap starting.
I20260811 17:18:27.537768 13438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:27.538687 13438 log.cc:824] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:27.541505 13438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3: No bootstrap required, opened a new log
I20260811 17:18:27.541817 13441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:27.533120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50661 (local address 127.0.0.1:38617)
0811 17:18:27.533736 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:27.533754 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:27.533795 (+ 41us) server_negotiation.cc:407] Connection header received
0811 17:18:27.534469 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:27.534494 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:27.534886 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:27.535304 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:27.535784 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:27.536928 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:27.537912 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:27.538436 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:27.538636 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:27.539681 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:27.539690 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:27.539709 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:27.540183 (+ 474us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:27.540243 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:27.540266 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:27.540364 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:27.540644 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 17:18:27.540905 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":174,"threads_started":1}
I20260811 17:18:27.545246 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:27.546726 13438 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5c8833c3384d57a55f08a24dd461b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38617 } }
I20260811 17:18:27.547106 13438 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:27.547180 13438 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5c8833c3384d57a55f08a24dd461b3, State: Initialized, Role: FOLLOWER
I20260811 17:18:27.547706 13438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5c8833c3384d57a55f08a24dd461b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38617 } }
I20260811 17:18:27.547902 13438 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:27.547992 13438 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:27.548151 13438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:27.549557 13438 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5c8833c3384d57a55f08a24dd461b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38617 } }
I20260811 17:18:27.549985 13438 leader_election.cc:302] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e5c8833c3384d57a55f08a24dd461b3; no voters:
I20260811 17:18:27.550438 13438 leader_election.cc:288] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:27.550549 13448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:27.550747 13448 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3 [term 1 LEADER]: Becoming Leader. State: Replica: 3e5c8833c3384d57a55f08a24dd461b3, State: Running, Role: LEADER
I20260811 17:18:27.551091 13448 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5c8833c3384d57a55f08a24dd461b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38617 } }
I20260811 17:18:27.552012 13438 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:27.552709 13441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:27.547956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37137 (local address 127.0.0.1:38617)
0811 17:18:27.548096 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:27.548101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:27.548348 (+ 247us) server_negotiation.cc:407] Connection header received
0811 17:18:27.548561 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:27.548564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:27.548621 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:27.548698 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:27.549070 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:27.549630 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:27.550683 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:27.551048 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:27.551150 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:27.552206 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:27.552210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:27.552211 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:27.552224 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:27.552253 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:27.552264 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:27.552349 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:27.552448 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 17:18:27.552534 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 17:18:27.554210 13450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e5c8833c3384d57a55f08a24dd461b3. Latest consensus state: current_term: 1 leader_uuid: "3e5c8833c3384d57a55f08a24dd461b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5c8833c3384d57a55f08a24dd461b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38617 } } }
I20260811 17:18:27.554378 13450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:27.554798 13451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e5c8833c3384d57a55f08a24dd461b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5c8833c3384d57a55f08a24dd461b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38617 } } }
I20260811 17:18:27.555194 13451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3 [sys.catalog]: This master's current role is: LEADER
W20260811 17:18:27.557579 13469 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:18:27.557677 13469 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:18:27.557751 13468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:27.558712 13468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:27.565186 13468 catalog_manager.cc:1347] Generated new cluster ID: d31c34f6282e49ffa1a6cf51d67c02c8
I20260811 17:18:27.565310 13468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:27.584198 13468 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:27.585363 13468 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:27.589887 13468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e5c8833c3384d57a55f08a24dd461b3: Generated new TSK 0
I20260811 17:18:27.590687 13468 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:27.604768 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 13360
2026-08-11T17:18:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1575 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:27 dist-test-slave-fd7l krb5kdc[13477](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:27 dist-test-slave-fd7l krb5kdc[13477](info): set up 2 sockets
Aug 11 17:18:27 dist-test-slave-fd7l krb5kdc[13477](info): commencing operation
krb5kdc: starting...
W20260811 17:18:28.762116 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:28 dist-test-slave-fd7l krb5kdc[13477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:18:28.904991 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45233
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:29.026633 13491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:29.027109 13491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:29.027204 13491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:29.032060 13491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:29.032192 13491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:29.032233 13491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:29.032274 13491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:29.032310 13491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:29.038343 13491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:29.039665 13491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:29.040846 13491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:29.050819 13497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:29.050968 13491 server_base.cc:1034] running on GCE node
W20260811 17:18:29.050851 13502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:29.051231 13498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:29.051908 13491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:29.053423 13491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:29.054633 13491 hybrid_clock.cc:648] HybridClock initialized: now 1786468709054620 us; error 31 us; skew 500 ppm
Aug 11 17:18:29 dist-test-slave-fd7l krb5kdc[13477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468709, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:29.057893 13491 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:18:29.058998 13491 webserver.cc:460] Webserver started at http://127.0.0.1:43587/ using document root <none> and password file <none>
I20260811 17:18:29.059665 13491 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:29.059774 13491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:29.060056 13491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:29.062536 13491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "c820def510e64ddfaaadd5ee673cdddc"
format_stamp: "Formatted at 2026-08-11 17:18:29 on dist-test-slave-fd7l"
server_key: "0ef36ce57c107bfdbf66560a504a4cf2"
server_key_iv: "96c3c27281e1d046ea3e93f2d880dcb6"
server_key_version: "encryptionkey@0"
I20260811 17:18:29.063233 13491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "c820def510e64ddfaaadd5ee673cdddc"
format_stamp: "Formatted at 2026-08-11 17:18:29 on dist-test-slave-fd7l"
server_key: "0ef36ce57c107bfdbf66560a504a4cf2"
server_key_iv: "96c3c27281e1d046ea3e93f2d880dcb6"
server_key_version: "encryptionkey@0"
I20260811 17:18:29.067950 13491 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 17:18:29.071288 13509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:29.072834 13491 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:29.073014 13491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "c820def510e64ddfaaadd5ee673cdddc"
format_stamp: "Formatted at 2026-08-11 17:18:29 on dist-test-slave-fd7l"
server_key: "0ef36ce57c107bfdbf66560a504a4cf2"
server_key_iv: "96c3c27281e1d046ea3e93f2d880dcb6"
server_key_version: "encryptionkey@0"
I20260811 17:18:29.073170 13491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:29.078692 13491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:29.082479 13491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:29.082737 13491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:29.090469 13491 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45233
I20260811 17:18:29.090477 13569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45233 every 8 connection(s)
I20260811 17:18:29.091641 13491 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:29.093358 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 13491
I20260811 17:18:29.093560 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:29.093892 21722 external_mini_cluster.cc:1428] Setting key 24d946cf563a51d7954c7c207a6066d8
I20260811 17:18:29.095072 13570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:29.100196 13573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:29.095543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35845 (local address 127.0.0.1:45233)
0811 17:18:29.096020 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:29.096049 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:29.096093 (+ 44us) server_negotiation.cc:407] Connection header received
0811 17:18:29.096993 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:29.097017 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:29.097394 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:29.097780 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:29.098690 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:29.098697 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:29.098714 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:29.098727 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:29.098799 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:29.098830 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:29.098982 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:29.099068 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 17:18:29.099231 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":133,"threads_started":1}
I20260811 17:18:29.102699 13570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc: Bootstrap starting.
I20260811 17:18:29.103500 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:29.105248 13570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:29.106125 13570 log.cc:824] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:29.108007 13573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:29.105998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58181 (local address 127.0.0.1:45233)
0811 17:18:29.106157 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:29.106162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:29.106396 (+ 234us) server_negotiation.cc:407] Connection header received
0811 17:18:29.106541 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:29.106543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:29.106607 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:29.106713 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:29.107498 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:29.107502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:29.107504 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:29.107517 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:29.107624 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:29.107641 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:29.107737 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:29.107835 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 17:18:29.107875 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 17:18:29.108753 13570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc: No bootstrap required, opened a new log
I20260811 17:18:29.112386 13570 raft_consensus.cc:348] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c820def510e64ddfaaadd5ee673cdddc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45233 } }
I20260811 17:18:29.112730 13570 raft_consensus.cc:374] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:29.112819 13570 raft_consensus.cc:729] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c820def510e64ddfaaadd5ee673cdddc, State: Initialized, Role: FOLLOWER
I20260811 17:18:29.113296 13570 consensus_queue.cc:260] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c820def510e64ddfaaadd5ee673cdddc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45233 } }
I20260811 17:18:29.113456 13570 raft_consensus.cc:388] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:29.113547 13570 raft_consensus.cc:482] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:29.113658 13570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:29.114711 13570 raft_consensus.cc:504] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c820def510e64ddfaaadd5ee673cdddc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45233 } }
I20260811 17:18:29.115103 13570 leader_election.cc:302] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c820def510e64ddfaaadd5ee673cdddc; no voters:
I20260811 17:18:29.115482 13570 leader_election.cc:288] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:29.115574 13581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:29.115772 13581 raft_consensus.cc:686] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc [term 1 LEADER]: Becoming Leader. State: Replica: c820def510e64ddfaaadd5ee673cdddc, State: Running, Role: LEADER
I20260811 17:18:29.116086 13581 consensus_queue.cc:237] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c820def510e64ddfaaadd5ee673cdddc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45233 } }
I20260811 17:18:29.116945 13570 sys_catalog.cc:564] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:29.118101 13584 sys_catalog.cc:455] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c820def510e64ddfaaadd5ee673cdddc. Latest consensus state: current_term: 1 leader_uuid: "c820def510e64ddfaaadd5ee673cdddc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c820def510e64ddfaaadd5ee673cdddc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45233 } } }
I20260811 17:18:29.118240 13584 sys_catalog.cc:458] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:29.118589 13582 sys_catalog.cc:455] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c820def510e64ddfaaadd5ee673cdddc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c820def510e64ddfaaadd5ee673cdddc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45233 } } }
I20260811 17:18:29.118701 13591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:29.118734 13582 sys_catalog.cc:458] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:29.122270 13591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:29.128966 13591 catalog_manager.cc:1347] Generated new cluster ID: 4f0d41abad9c4deda476baf1521eab99
I20260811 17:18:29.129073 13591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:29.164433 13591 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:29.165035 13591 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:29.173353 13591 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c820def510e64ddfaaadd5ee673cdddc: Generated new TSK 0
I20260811 17:18:29.174226 13591 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:29.193138 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 13491
2026-08-11T17:18:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1588 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:29 dist-test-slave-fd7l krb5kdc[13606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:29 dist-test-slave-fd7l krb5kdc[13606](info): set up 2 sockets
Aug 11 17:18:29 dist-test-slave-fd7l krb5kdc[13606](info): commencing operation
krb5kdc: starting...
W20260811 17:18:30.447770 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:30 dist-test-slave-fd7l krb5kdc[13606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:18:30.592810 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45395
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:30.709487 13624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:30.709793 13624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:30.709883 13624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:30.713960 13624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:30.714057 13624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:30.714097 13624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:30.714128 13624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:30.714152 13624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:30.722976 13624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:30.724239 13624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:30.725365 13624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:30.734498 13630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:30.734477 13629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:30.734477 13632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:30.734871 13624 server_base.cc:1034] running on GCE node
I20260811 17:18:30.735467 13624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:30.736665 13624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:30.737857 13624 hybrid_clock.cc:648] HybridClock initialized: now 1786468710737826 us; error 42 us; skew 500 ppm
Aug 11 17:18:30 dist-test-slave-fd7l krb5kdc[13606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468710, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:30.745036 13624 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:18:30.746080 13624 webserver.cc:460] Webserver started at http://127.0.0.1:43833/ using document root <none> and password file <none>
I20260811 17:18:30.746704 13624 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:30.746804 13624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:30.747052 13624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:30.749284 13624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "f6bf65bc657c444e919b37061165797e"
format_stamp: "Formatted at 2026-08-11 17:18:30 on dist-test-slave-fd7l"
server_key: "b4f2c2bcfc9c51085069f10ec44d30fa"
server_key_iv: "421bda64e3397c57d2c8229dcab8ff1f"
server_key_version: "encryptionkey@0"
I20260811 17:18:30.750044 13624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "f6bf65bc657c444e919b37061165797e"
format_stamp: "Formatted at 2026-08-11 17:18:30 on dist-test-slave-fd7l"
server_key: "b4f2c2bcfc9c51085069f10ec44d30fa"
server_key_iv: "421bda64e3397c57d2c8229dcab8ff1f"
server_key_version: "encryptionkey@0"
I20260811 17:18:30.754021 13624 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:18:30.756975 13639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:30.758527 13624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:18:30.758677 13624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "f6bf65bc657c444e919b37061165797e"
format_stamp: "Formatted at 2026-08-11 17:18:30 on dist-test-slave-fd7l"
server_key: "b4f2c2bcfc9c51085069f10ec44d30fa"
server_key_iv: "421bda64e3397c57d2c8229dcab8ff1f"
server_key_version: "encryptionkey@0"
I20260811 17:18:30.758845 13624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:30.774238 13624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:30.777514 13624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:30.777725 13624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:30.785383 13624 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45395
I20260811 17:18:30.785396 13697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45395 every 8 connection(s)
I20260811 17:18:30.786410 13624 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:30.789331 13698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:30.791293 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 13624
I20260811 17:18:30.791446 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:30.791771 21722 external_mini_cluster.cc:1428] Setting key 9ed8e896d6b67b227a43db24ee671ad0
I20260811 17:18:30.796350 13698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e: Bootstrap starting.
I20260811 17:18:30.799057 13698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:30.800019 13698 log.cc:824] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e: Log is configured to *not* fsync() on all Append() calls
Aug 11 17:18:30 dist-test-slave-fd7l krb5kdc[13606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 17:18:30.802390 13698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e: No bootstrap required, opened a new log
I20260811 17:18:30.807420 13701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:30.793291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:50971 (local address 127.0.0.1:45395)
0811 17:18:30.793713 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:30.793739 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:30.793777 (+ 38us) server_negotiation.cc:407] Connection header received
0811 17:18:30.794419 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:30.794436 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:30.794778 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:30.795065 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:30.796048 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:30.797152 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:30.798424 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:30.798907 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:30.799057 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:30.801892 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:30.801899 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:30.801917 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:30.801949 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:30.804044 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:30.804600 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:30.804608 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:30.804614 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:30.804774 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:30.805132 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:30.805138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:30.805142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:30.805698 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:30.805904 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:30.806179 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 17:18:30.806413 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":152,"threads_started":1}
I20260811 17:18:30.807345 13698 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6bf65bc657c444e919b37061165797e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45395 } }
I20260811 17:18:30.807612 13698 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:30.807677 13698 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bf65bc657c444e919b37061165797e, State: Initialized, Role: FOLLOWER
I20260811 17:18:30.808171 13698 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6bf65bc657c444e919b37061165797e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45395 } }
I20260811 17:18:30.808408 13698 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:30.808490 13698 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:30.808645 13698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:30.809825 13698 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6bf65bc657c444e919b37061165797e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45395 } }
I20260811 17:18:30.810153 13698 leader_election.cc:302] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6bf65bc657c444e919b37061165797e; no voters:
I20260811 17:18:30.810534 13698 leader_election.cc:288] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:30.810645 13703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:30.810858 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:30.810936 13703 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e [term 1 LEADER]: Becoming Leader. State: Replica: f6bf65bc657c444e919b37061165797e, State: Running, Role: LEADER
I20260811 17:18:30.811254 13703 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6bf65bc657c444e919b37061165797e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45395 } }
I20260811 17:18:30.811993 13698 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:30.812723 13704 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6bf65bc657c444e919b37061165797e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6bf65bc657c444e919b37061165797e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45395 } } }
I20260811 17:18:30.812855 13704 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:30.813087 13705 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6bf65bc657c444e919b37061165797e. Latest consensus state: current_term: 1 leader_uuid: "f6bf65bc657c444e919b37061165797e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6bf65bc657c444e919b37061165797e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45395 } } }
I20260811 17:18:30.813349 13705 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:30.813781 13715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:30.817016 13715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:30.823030 13715 catalog_manager.cc:1347] Generated new cluster ID: ca034ab40f46464cb82d65220e236ea4
I20260811 17:18:30.823093 13715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:30.826208 13701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:30.813419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:49827 (local address 127.0.0.1:45395)
0811 17:18:30.813526 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:30.813530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:30.816848 (+ 3318us) server_negotiation.cc:407] Connection header received
0811 17:18:30.817059 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:30.817064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:30.817122 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:30.817218 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:30.818388 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:30.819144 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:30.820344 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:30.820704 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:30.820809 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:30.822294 (+ 1485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:30.822298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:30.822300 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:30.822333 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:30.824101 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:30.824656 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:30.824660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:30.824661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:30.824737 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:30.825060 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:30.825063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:30.825065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:30.825245 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:30.825386 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:30.825964 (+ 578us) server_negotiation.cc:299] Negotiation successful
0811 17:18:30.826061 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 17:18:30.853122 13715 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:30.854218 13715 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:30.859067 13715 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6bf65bc657c444e919b37061165797e: Generated new TSK 0
I20260811 17:18:30.859874 13715 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:30.864601 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 13624
W20260811 17:18:30.873255 13602 connection.cc:535] client connection to 127.0.0.1:45395 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45395): Connection reset by peer (error 104)
2026-08-11T17:18:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1669 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:30 dist-test-slave-fd7l krb5kdc[13731](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:30 dist-test-slave-fd7l krb5kdc[13731](info): set up 2 sockets
Aug 11 17:18:30 dist-test-slave-fd7l krb5kdc[13731](info): commencing operation
krb5kdc: starting...
W20260811 17:18:31.995896 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:32 dist-test-slave-fd7l krb5kdc[13731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:18:32.136539 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43243
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:32.256028 13747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:32.256345 13747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:32.256390 13747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:32.260707 13747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:32.260790 13747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:32.260811 13747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:32.260828 13747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:32.260844 13747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:32.266567 13747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:32.267782 13747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:32.268949 13747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:32.278358 13753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:32.278358 13756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:32.278410 13752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:32.278638 13747 server_base.cc:1034] running on GCE node
I20260811 17:18:32.279366 13747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:32.280705 13747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:32.281893 13747 hybrid_clock.cc:648] HybridClock initialized: now 1786468712281879 us; error 27 us; skew 500 ppm
Aug 11 17:18:32 dist-test-slave-fd7l krb5kdc[13731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468712, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:32.285286 13747 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:18:32.286517 13747 webserver.cc:460] Webserver started at http://127.0.0.1:35993/ using document root <none> and password file <none>
I20260811 17:18:32.287206 13747 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:32.287293 13747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:32.287569 13747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:32.290096 13747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "b713be856c1a4eb68a0fcdfefd213107"
format_stamp: "Formatted at 2026-08-11 17:18:32 on dist-test-slave-fd7l"
server_key: "50880e4389a9aa4eec68cec467c1c86a"
server_key_iv: "f9d5f3a49872353c50d85bbc3c60f16c"
server_key_version: "encryptionkey@0"
I20260811 17:18:32.290781 13747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "b713be856c1a4eb68a0fcdfefd213107"
format_stamp: "Formatted at 2026-08-11 17:18:32 on dist-test-slave-fd7l"
server_key: "50880e4389a9aa4eec68cec467c1c86a"
server_key_iv: "f9d5f3a49872353c50d85bbc3c60f16c"
server_key_version: "encryptionkey@0"
I20260811 17:18:32.295392 13747 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:18:32.298640 13768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:32.300326 13747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:18:32.300509 13747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "b713be856c1a4eb68a0fcdfefd213107"
format_stamp: "Formatted at 2026-08-11 17:18:32 on dist-test-slave-fd7l"
server_key: "50880e4389a9aa4eec68cec467c1c86a"
server_key_iv: "f9d5f3a49872353c50d85bbc3c60f16c"
server_key_version: "encryptionkey@0"
I20260811 17:18:32.300743 13747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:32.321641 13747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:32.325847 13747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:32.326110 13747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:32.334527 13747 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43243
I20260811 17:18:32.334563 13827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43243 every 8 connection(s)
I20260811 17:18:32.335687 13747 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:32.338912 13829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:32.345155 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 13747
I20260811 17:18:32.345280 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:32.345350 13829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107: Bootstrap starting.
I20260811 17:18:32.345656 21722 external_mini_cluster.cc:1428] Setting key 7aa22469a3838064c642e4ee4debe240
I20260811 17:18:32.348256 13829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:32.349197 13829 log.cc:824] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:32.351363 13829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107: No bootstrap required, opened a new log
I20260811 17:18:32.355672 13833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:32.347180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:54429 (local address 127.0.0.1:43243)
0811 17:18:32.347726 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:32.347742 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:32.347779 (+ 37us) server_negotiation.cc:407] Connection header received
0811 17:18:32.348629 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:32.348651 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:32.349036 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:32.349391 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:32.349827 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:32.350815 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:32.351828 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:32.352328 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:32.352508 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:32.353672 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:32.353678 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:32.353693 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:32.354158 (+ 465us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:32.354222 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:32.354248 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:32.354417 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:32.354647 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 17:18:32.354811 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":182,"threads_started":1}
I20260811 17:18:32.355723 13829 raft_consensus.cc:348] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b713be856c1a4eb68a0fcdfefd213107" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43243 } }
I20260811 17:18:32.356086 13829 raft_consensus.cc:374] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:32.356180 13829 raft_consensus.cc:729] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b713be856c1a4eb68a0fcdfefd213107, State: Initialized, Role: FOLLOWER
I20260811 17:18:32.356753 13829 consensus_queue.cc:260] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b713be856c1a4eb68a0fcdfefd213107" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43243 } }
I20260811 17:18:32.356935 13829 raft_consensus.cc:388] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:32.357046 13829 raft_consensus.cc:482] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:32.357165 13829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:32.358566 13829 raft_consensus.cc:504] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b713be856c1a4eb68a0fcdfefd213107" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43243 } }
I20260811 17:18:32.359046 13829 leader_election.cc:302] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b713be856c1a4eb68a0fcdfefd213107; no voters:
I20260811 17:18:32.359464 13829 leader_election.cc:288] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:32.359584 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:32.359583 13835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:32.359824 13835 raft_consensus.cc:686] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107 [term 1 LEADER]: Becoming Leader. State: Replica: b713be856c1a4eb68a0fcdfefd213107, State: Running, Role: LEADER
I20260811 17:18:32.360242 13835 consensus_queue.cc:237] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b713be856c1a4eb68a0fcdfefd213107" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43243 } }
I20260811 17:18:32.360940 13829 sys_catalog.cc:564] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:32.362082 13837 sys_catalog.cc:455] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b713be856c1a4eb68a0fcdfefd213107. Latest consensus state: current_term: 1 leader_uuid: "b713be856c1a4eb68a0fcdfefd213107" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b713be856c1a4eb68a0fcdfefd213107" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43243 } } }
I20260811 17:18:32.362299 13836 sys_catalog.cc:455] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b713be856c1a4eb68a0fcdfefd213107" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b713be856c1a4eb68a0fcdfefd213107" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43243 } } }
I20260811 17:18:32.362344 13837 sys_catalog.cc:458] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:32.362404 13836 sys_catalog.cc:458] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:32.363327 13851 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:32.367008 13851 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:32.368101 13833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:32.362645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:38651 (local address 127.0.0.1:43243)
0811 17:18:32.362779 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:32.362784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:32.362920 (+ 136us) server_negotiation.cc:407] Connection header received
0811 17:18:32.363089 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:32.363092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:32.363154 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:32.363242 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:32.363780 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:32.364638 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:32.365925 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:32.366302 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:32.366442 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:32.367505 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:32.367510 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:32.367513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:32.367531 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:32.367565 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:32.367583 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:32.367699 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:32.367908 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 17:18:32.367951 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 17:18:32.373584 13851 catalog_manager.cc:1347] Generated new cluster ID: c911c1391e614899814edcf4749de739
I20260811 17:18:32.373677 13851 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:32.379869 13851 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:32.381289 13851 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:32.394727 13851 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b713be856c1a4eb68a0fcdfefd213107: Generated new TSK 0
I20260811 17:18:32.395718 13851 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:32.406847 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 13747
W20260811 17:18:32.417953 13729 connection.cc:535] client connection to 127.0.0.1:43243 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43243): Connection reset by peer (error 104)
2026-08-11T17:18:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1544 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:32 dist-test-slave-fd7l krb5kdc[13864](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:32 dist-test-slave-fd7l krb5kdc[13864](info): set up 2 sockets
Aug 11 17:18:32 dist-test-slave-fd7l krb5kdc[13864](info): commencing operation
krb5kdc: starting...
W20260811 17:18:33.551858 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:33 dist-test-slave-fd7l krb5kdc[13864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:18:33.696892 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44147
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:33.821620 13878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:33.821979 13878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:33.822064 13878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:33.826339 13878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:33.826442 13878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:33.826488 13878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:33.826550 13878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:33.826589 13878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:33.832305 13878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:33.833547 13878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:33.834635 13878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:18:33.843973 13878 server_base.cc:1034] running on GCE node
W20260811 17:18:33.843915 13883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:33.843910 13884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:33.843904 13887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:33.844973 13878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:33.846168 13878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:33.847381 13878 hybrid_clock.cc:648] HybridClock initialized: now 1786468713847345 us; error 51 us; skew 500 ppm
Aug 11 17:18:33 dist-test-slave-fd7l krb5kdc[13864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468713, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:33.850589 13878 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:18:33.851541 13878 webserver.cc:460] Webserver started at http://127.0.0.1:43017/ using document root <none> and password file <none>
I20260811 17:18:33.852177 13878 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:33.852277 13878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:33.852584 13878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:33.854766 13878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "33d6c805b9554138b59e03f2c8e1cb6a"
format_stamp: "Formatted at 2026-08-11 17:18:33 on dist-test-slave-fd7l"
server_key: "0719c05d4b86ede1eadd8907b24fd6d0"
server_key_iv: "c57df0f598a8ee5b616a7823db3c9419"
server_key_version: "encryptionkey@0"
I20260811 17:18:33.855384 13878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "33d6c805b9554138b59e03f2c8e1cb6a"
format_stamp: "Formatted at 2026-08-11 17:18:33 on dist-test-slave-fd7l"
server_key: "0719c05d4b86ede1eadd8907b24fd6d0"
server_key_iv: "c57df0f598a8ee5b616a7823db3c9419"
server_key_version: "encryptionkey@0"
I20260811 17:18:33.859516 13878 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:18:33.862682 13896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:33.864126 13878 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:18:33.864323 13878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "33d6c805b9554138b59e03f2c8e1cb6a"
format_stamp: "Formatted at 2026-08-11 17:18:33 on dist-test-slave-fd7l"
server_key: "0719c05d4b86ede1eadd8907b24fd6d0"
server_key_iv: "c57df0f598a8ee5b616a7823db3c9419"
server_key_version: "encryptionkey@0"
I20260811 17:18:33.864467 13878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:33.869503 13878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:33.872819 13878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:33.873055 13878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:33.880589 13878 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44147
I20260811 17:18:33.880630 13956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44147 every 8 connection(s)
I20260811 17:18:33.881614 13878 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:33.884682 13957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:33.885599 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 13878
I20260811 17:18:33.885753 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:33.886083 21722 external_mini_cluster.cc:1428] Setting key 2d33ea7761acc7cbc0f7a32d9865fcfa
I20260811 17:18:33.892026 13960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:33.887631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:56793 (local address 127.0.0.1:44147)
0811 17:18:33.888095 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:33.888110 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:33.888148 (+ 38us) server_negotiation.cc:407] Connection header received
0811 17:18:33.888837 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:33.888864 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:33.889180 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:33.889480 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:33.890487 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:33.890494 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:33.890509 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:33.890521 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:33.890585 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:33.890607 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:33.890738 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:33.890981 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 17:18:33.891218 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":137,"threads_started":1}
I20260811 17:18:33.892385 13957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a: Bootstrap starting.
I20260811 17:18:33.895007 13957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:33.895109 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:33.896083 13957 log.cc:824] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:33.898789 13957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a: No bootstrap required, opened a new log
I20260811 17:18:33.899694 13960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:33.897833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:59903 (local address 127.0.0.1:44147)
0811 17:18:33.897984 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:33.897989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:33.898168 (+ 179us) server_negotiation.cc:407] Connection header received
0811 17:18:33.898263 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:33.898266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:33.898307 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:33.898406 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:33.899254 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:33.899258 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:33.899261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:33.899275 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:33.899307 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:33.899321 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:33.899423 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:33.899515 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 17:18:33.899568 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 17:18:33.903393 13957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d6c805b9554138b59e03f2c8e1cb6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44147 } }
I20260811 17:18:33.903666 13957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:33.903713 13957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d6c805b9554138b59e03f2c8e1cb6a, State: Initialized, Role: FOLLOWER
I20260811 17:18:33.904204 13957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d6c805b9554138b59e03f2c8e1cb6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44147 } }
I20260811 17:18:33.904349 13957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:33.904404 13957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:33.904488 13957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:33.905715 13957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d6c805b9554138b59e03f2c8e1cb6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44147 } }
I20260811 17:18:33.906141 13957 leader_election.cc:302] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33d6c805b9554138b59e03f2c8e1cb6a; no voters:
I20260811 17:18:33.906447 13957 leader_election.cc:288] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:33.906606 13967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:33.906879 13967 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a [term 1 LEADER]: Becoming Leader. State: Replica: 33d6c805b9554138b59e03f2c8e1cb6a, State: Running, Role: LEADER
I20260811 17:18:33.907267 13967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d6c805b9554138b59e03f2c8e1cb6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44147 } }
I20260811 17:18:33.907883 13957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:33.909303 13968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33d6c805b9554138b59e03f2c8e1cb6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d6c805b9554138b59e03f2c8e1cb6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44147 } } }
I20260811 17:18:33.909487 13968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:33.909297 13969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33d6c805b9554138b59e03f2c8e1cb6a. Latest consensus state: current_term: 1 leader_uuid: "33d6c805b9554138b59e03f2c8e1cb6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33d6c805b9554138b59e03f2c8e1cb6a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44147 } } }
I20260811 17:18:33.909682 13969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:33.910497 13977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:33.913998 13977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:33.920244 13977 catalog_manager.cc:1347] Generated new cluster ID: 7b4358a561504b3f96c003a87d87e5bf
I20260811 17:18:33.920393 13977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:33.943444 13977 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:33.943881 13977 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:33.952956 13977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33d6c805b9554138b59e03f2c8e1cb6a: Generated new TSK 0
I20260811 17:18:33.953819 13977 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:33.960427 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 13878
2026-08-11T17:18:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1554 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:34 dist-test-slave-fd7l krb5kdc[13991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:34 dist-test-slave-fd7l krb5kdc[13991](info): set up 2 sockets
Aug 11 17:18:34 dist-test-slave-fd7l krb5kdc[13991](info): commencing operation
krb5kdc: starting...
W20260811 17:18:35.115074 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:35 dist-test-slave-fd7l krb5kdc[13991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:18:35.256409 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34767
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34767
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:35.384769 14007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:35.385077 14007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:35.385121 14007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:35.389382 14007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:35.389464 14007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:35.389485 14007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:35.389503 14007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:35.389518 14007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:35.395300 14007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34767
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:35.396631 14007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:35.398288 14007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:35.408435 14013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:35.408450 14012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:35.408420 14015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:35.408746 14007 server_base.cc:1034] running on GCE node
I20260811 17:18:35.409581 14007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:35.410974 14007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:35.412201 14007 hybrid_clock.cc:648] HybridClock initialized: now 1786468715412181 us; error 35 us; skew 500 ppm
Aug 11 17:18:35 dist-test-slave-fd7l krb5kdc[13991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468715, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:35.415516 14007 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:18:35.416495 14007 webserver.cc:460] Webserver started at http://127.0.0.1:46315/ using document root <none> and password file <none>
I20260811 17:18:35.417189 14007 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:35.417255 14007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:35.417440 14007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:35.419675 14007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "b7fb20d0f8604d9193ee6012e34afa26"
format_stamp: "Formatted at 2026-08-11 17:18:35 on dist-test-slave-fd7l"
server_key: "659ed26f079d47140f68e9f849179281"
server_key_iv: "954b243e8633b659363fe0f6199c85a1"
server_key_version: "encryptionkey@0"
I20260811 17:18:35.420243 14007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "b7fb20d0f8604d9193ee6012e34afa26"
format_stamp: "Formatted at 2026-08-11 17:18:35 on dist-test-slave-fd7l"
server_key: "659ed26f079d47140f68e9f849179281"
server_key_iv: "954b243e8633b659363fe0f6199c85a1"
server_key_version: "encryptionkey@0"
I20260811 17:18:35.424502 14007 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 17:18:35.427357 14024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:35.428915 14007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:35.429041 14007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "b7fb20d0f8604d9193ee6012e34afa26"
format_stamp: "Formatted at 2026-08-11 17:18:35 on dist-test-slave-fd7l"
server_key: "659ed26f079d47140f68e9f849179281"
server_key_iv: "954b243e8633b659363fe0f6199c85a1"
server_key_version: "encryptionkey@0"
I20260811 17:18:35.429136 14007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:35.440373 14007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:35.443928 14007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:35.444110 14007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:35.452730 14007 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34767
I20260811 17:18:35.452762 14084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34767 every 8 connection(s)
I20260811 17:18:35.453886 14007 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:35.455646 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 14007
I20260811 17:18:35.455833 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:35.456239 21722 external_mini_cluster.cc:1428] Setting key 4fb4f8452db76d3e2542c3d2633db8ab
I20260811 17:18:35.457710 14085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:35.465180 14085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26: Bootstrap starting.
Aug 11 17:18:35 dist-test-slave-fd7l krb5kdc[13991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 17:18:35.468912 14085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:35.470039 14085 log.cc:824] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:35.472833 14085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26: No bootstrap required, opened a new log
I20260811 17:18:35.474931 14088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:35.457897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:36857 (local address 127.0.0.1:34767)
0811 17:18:35.458407 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:35.458424 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:35.458463 (+ 39us) server_negotiation.cc:407] Connection header received
0811 17:18:35.459180 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:35.459197 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:35.459554 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:35.459931 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:35.461091 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:35.462110 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:35.463566 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:35.464437 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:35.465200 (+ 763us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:35.468816 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:35.468823 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:35.468846 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:35.468883 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:35.471276 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:35.471982 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:35.471987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:35.471993 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:35.472123 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:35.472658 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:35.472661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:35.472662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:35.473196 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:35.473377 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:35.473801 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 17:18:35.474085 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":159,"threads_started":1}
I20260811 17:18:35.477485 14085 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7fb20d0f8604d9193ee6012e34afa26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34767 } }
I20260811 17:18:35.477735 14085 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:35.477825 14085 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fb20d0f8604d9193ee6012e34afa26, State: Initialized, Role: FOLLOWER
I20260811 17:18:35.478435 14085 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7fb20d0f8604d9193ee6012e34afa26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34767 } }
I20260811 17:18:35.478621 14085 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:35.478636 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:35.478720 14085 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:35.478835 14085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:35.479920 14085 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7fb20d0f8604d9193ee6012e34afa26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34767 } }
I20260811 17:18:35.480367 14085 leader_election.cc:302] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7fb20d0f8604d9193ee6012e34afa26; no voters:
I20260811 17:18:35.480767 14085 leader_election.cc:288] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:35.480882 14094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:35.481091 14094 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26 [term 1 LEADER]: Becoming Leader. State: Replica: b7fb20d0f8604d9193ee6012e34afa26, State: Running, Role: LEADER
I20260811 17:18:35.481434 14094 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7fb20d0f8604d9193ee6012e34afa26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34767 } }
I20260811 17:18:35.482337 14085 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:35.483349 14095 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7fb20d0f8604d9193ee6012e34afa26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7fb20d0f8604d9193ee6012e34afa26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34767 } } }
I20260811 17:18:35.483496 14095 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:35.483857 14097 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7fb20d0f8604d9193ee6012e34afa26. Latest consensus state: current_term: 1 leader_uuid: "b7fb20d0f8604d9193ee6012e34afa26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7fb20d0f8604d9193ee6012e34afa26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34767 } } }
I20260811 17:18:35.483929 14105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:35.484011 14097 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:35.487843 14105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:35.493431 14088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:35.481217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:54019 (local address 127.0.0.1:34767)
0811 17:18:35.481374 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:35.481381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:35.482072 (+ 691us) server_negotiation.cc:407] Connection header received
0811 17:18:35.482254 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:35.482258 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:35.482315 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:35.482398 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:35.485391 (+ 2993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:35.486180 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:35.487104 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:35.487475 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:35.487568 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:35.489493 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:35.489497 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:35.489500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:35.489527 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:35.491313 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:35.491949 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:35.491953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:35.491954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:35.492045 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:35.492468 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:35.492472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:35.492473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:35.492705 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:35.492867 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:35.493161 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 17:18:35.493260 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 17:18:35.495748 14105 catalog_manager.cc:1347] Generated new cluster ID: 88a6fe462b634c9fac766291224bacff
I20260811 17:18:35.495854 14105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:35.520757 14105 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:35.522080 14105 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:35.526921 14105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7fb20d0f8604d9193ee6012e34afa26: Generated new TSK 0
I20260811 17:18:35.527663 14105 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:35.534201 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 14007
W20260811 17:18:35.544399 13989 connection.cc:535] client connection to 127.0.0.1:34767 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34767): Connection reset by peer (error 104)
2026-08-11T17:18:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1571 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:35 dist-test-slave-fd7l krb5kdc[14119](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:35 dist-test-slave-fd7l krb5kdc[14119](info): set up 2 sockets
Aug 11 17:18:35 dist-test-slave-fd7l krb5kdc[14119](info): commencing operation
krb5kdc: starting...
W20260811 17:18:36.755630 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:36 dist-test-slave-fd7l krb5kdc[14119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:18:36.893494 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45411
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:37.013221 14133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:37.013578 14133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:37.013670 14133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:37.017870 14133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:37.017968 14133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:37.018015 14133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:37.018077 14133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:37.018118 14133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:37.023742 14133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45411
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:37.025137 14133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:37.026232 14133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:37.035466 14143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:37.035418 14140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:37.035498 14141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:37.035923 14133 server_base.cc:1034] running on GCE node
I20260811 17:18:37.036659 14133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:37.037894 14133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:37.039122 14133 hybrid_clock.cc:648] HybridClock initialized: now 1786468717039109 us; error 27 us; skew 500 ppm
Aug 11 17:18:37 dist-test-slave-fd7l krb5kdc[14119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468717, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:37.046900 14133 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:18:37.048007 14133 webserver.cc:460] Webserver started at http://127.0.0.1:36807/ using document root <none> and password file <none>
I20260811 17:18:37.048677 14133 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:37.048779 14133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:37.049023 14133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:37.051263 14133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "c1fea0ddb934415aa794603651d4e6b3"
format_stamp: "Formatted at 2026-08-11 17:18:37 on dist-test-slave-fd7l"
server_key: "9e6e105366bebc9ae578d1dc95b7e93e"
server_key_iv: "ba22c246bcee31ba8fda28fe2ab55237"
server_key_version: "encryptionkey@0"
I20260811 17:18:37.051874 14133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "c1fea0ddb934415aa794603651d4e6b3"
format_stamp: "Formatted at 2026-08-11 17:18:37 on dist-test-slave-fd7l"
server_key: "9e6e105366bebc9ae578d1dc95b7e93e"
server_key_iv: "ba22c246bcee31ba8fda28fe2ab55237"
server_key_version: "encryptionkey@0"
I20260811 17:18:37.055768 14133 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:18:37.058398 14150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:37.059871 14133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:18:37.060024 14133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "c1fea0ddb934415aa794603651d4e6b3"
format_stamp: "Formatted at 2026-08-11 17:18:37 on dist-test-slave-fd7l"
server_key: "9e6e105366bebc9ae578d1dc95b7e93e"
server_key_iv: "ba22c246bcee31ba8fda28fe2ab55237"
server_key_version: "encryptionkey@0"
I20260811 17:18:37.060150 14133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:37.071570 14133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:37.074755 14133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:37.074970 14133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:37.082547 14133 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45411
I20260811 17:18:37.082574 14210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45411 every 8 connection(s)
I20260811 17:18:37.083566 14133 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:37.086674 14211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:37.092427 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 14133
I20260811 17:18:37.092641 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:37.092666 14211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3: Bootstrap starting.
I20260811 17:18:37.092989 21722 external_mini_cluster.cc:1428] Setting key b4443a794c9496b0cf52fbf6bf9dc314
I20260811 17:18:37.095417 14211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:37.096225 14211 log.cc:824] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:37.098834 14211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3: No bootstrap required, opened a new log
I20260811 17:18:37.104004 14211 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1fea0ddb934415aa794603651d4e6b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45411 } }
I20260811 17:18:37.104336 14211 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:37.104404 14211 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1fea0ddb934415aa794603651d4e6b3, State: Initialized, Role: FOLLOWER
I20260811 17:18:37.104460 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:37.094744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:49891 (local address 127.0.0.1:45411)
0811 17:18:37.095246 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:37.095263 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:37.095303 (+ 40us) server_negotiation.cc:407] Connection header received
0811 17:18:37.096107 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:37.096130 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:37.096649 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:37.096971 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:37.097447 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:37.099037 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:37.099989 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:37.100793 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:37.100976 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:37.101974 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:37.101994 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:37.102018 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:37.102700 (+ 682us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:37.102787 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:37.102820 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:37.102989 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:37.103180 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 17:18:37.103341 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":175,"threads_started":1}
I20260811 17:18:37.104955 14211 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1fea0ddb934415aa794603651d4e6b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45411 } }
I20260811 17:18:37.105149 14211 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:37.105227 14211 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:37.105331 14211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:37.106536 14211 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1fea0ddb934415aa794603651d4e6b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45411 } }
I20260811 17:18:37.106945 14211 leader_election.cc:302] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1fea0ddb934415aa794603651d4e6b3; no voters:
I20260811 17:18:37.107425 14211 leader_election.cc:288] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:37.107473 14216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:37.107738 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:37.107749 14216 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3 [term 1 LEADER]: Becoming Leader. State: Replica: c1fea0ddb934415aa794603651d4e6b3, State: Running, Role: LEADER
I20260811 17:18:37.108086 14216 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1fea0ddb934415aa794603651d4e6b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45411 } }
I20260811 17:18:37.108955 14211 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:37.109433 14218 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1fea0ddb934415aa794603651d4e6b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1fea0ddb934415aa794603651d4e6b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45411 } } }
I20260811 17:18:37.109558 14218 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:37.110284 14217 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1fea0ddb934415aa794603651d4e6b3. Latest consensus state: current_term: 1 leader_uuid: "c1fea0ddb934415aa794603651d4e6b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1fea0ddb934415aa794603651d4e6b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45411 } } }
I20260811 17:18:37.110391 14217 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:37.110766 14226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:37.114347 14226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:37.120289 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:37.110387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:58351 (local address 127.0.0.1:45411)
0811 17:18:37.110515 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:37.110521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:37.111746 (+ 1225us) server_negotiation.cc:407] Connection header received
0811 17:18:37.111966 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:37.111969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:37.112024 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:37.112126 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:37.112538 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:37.113312 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:37.114715 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:37.115103 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:37.115198 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:37.119443 (+ 4245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:37.119447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:37.119450 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:37.119470 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:37.119508 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:37.119523 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:37.119716 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:37.120100 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 17:18:37.120142 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 17:18:37.121649 14226 catalog_manager.cc:1347] Generated new cluster ID: d905c012e8ee4fb9b3f4477d1242040e
I20260811 17:18:37.121779 14226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:37.147785 14226 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:37.149148 14226 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:37.162663 14226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1fea0ddb934415aa794603651d4e6b3: Generated new TSK 0
I20260811 17:18:37.163551 14226 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:37.168327 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 14133
W20260811 17:18:37.178792 14117 connection.cc:535] client connection to 127.0.0.1:45411 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45411): Connection reset by peer (error 104)
2026-08-11T17:18:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1634 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:37 dist-test-slave-fd7l krb5kdc[14246](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:37 dist-test-slave-fd7l krb5kdc[14246](info): set up 2 sockets
Aug 11 17:18:37 dist-test-slave-fd7l krb5kdc[14246](info): commencing operation
krb5kdc: starting...
W20260811 17:18:38.306613 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:38 dist-test-slave-fd7l krb5kdc[14246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:18:38.448354 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33993
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:38.577049 14261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:38.577369 14261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:38.577414 14261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:38.581525 14261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:38.581593 14261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:38.581614 14261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:38.581630 14261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:38.581646 14261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:38.587317 14261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33993
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:38.588500 14261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:38.589704 14261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:38.599224 14269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:38.599234 14267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:38.599404 14261 server_base.cc:1034] running on GCE node
W20260811 17:18:38.599238 14266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:38.600266 14261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:38.601591 14261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:38.602821 14261 hybrid_clock.cc:648] HybridClock initialized: now 1786468718602790 us; error 62 us; skew 500 ppm
Aug 11 17:18:38 dist-test-slave-fd7l krb5kdc[14246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468718, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:38.606115 14261 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:18:38.607206 14261 webserver.cc:460] Webserver started at http://127.0.0.1:36925/ using document root <none> and password file <none>
I20260811 17:18:38.607817 14261 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:38.607914 14261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:38.608177 14261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:38.610852 14261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "b5c222c252e04cc1ba7f4b7a7e8d40a3"
format_stamp: "Formatted at 2026-08-11 17:18:38 on dist-test-slave-fd7l"
server_key: "677ca06c0aa37bbecd4afdb74ae8f9a5"
server_key_iv: "7b04f741b1222b6e618bf3980cf3aa8a"
server_key_version: "encryptionkey@0"
I20260811 17:18:38.611524 14261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "b5c222c252e04cc1ba7f4b7a7e8d40a3"
format_stamp: "Formatted at 2026-08-11 17:18:38 on dist-test-slave-fd7l"
server_key: "677ca06c0aa37bbecd4afdb74ae8f9a5"
server_key_iv: "7b04f741b1222b6e618bf3980cf3aa8a"
server_key_version: "encryptionkey@0"
I20260811 17:18:38.615859 14261 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:18:38.618741 14277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:38.620287 14261 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:18:38.620479 14261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "b5c222c252e04cc1ba7f4b7a7e8d40a3"
format_stamp: "Formatted at 2026-08-11 17:18:38 on dist-test-slave-fd7l"
server_key: "677ca06c0aa37bbecd4afdb74ae8f9a5"
server_key_iv: "7b04f741b1222b6e618bf3980cf3aa8a"
server_key_version: "encryptionkey@0"
I20260811 17:18:38.620671 14261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:38.625662 14261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:38.629335 14261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:38.629571 14261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:38.637684 14261 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33993
I20260811 17:18:38.637704 14337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33993 every 8 connection(s)
I20260811 17:18:38.638790 14261 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:38.642040 14338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:38.647311 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 14261
I20260811 17:18:38.647488 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:38.647802 21722 external_mini_cluster.cc:1428] Setting key 4d568a4620895194e760d79d60c2d38f
I20260811 17:18:38.648582 14338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3: Bootstrap starting.
I20260811 17:18:38.651299 14338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:38.652323 14338 log.cc:824] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:38.654068 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:38.649469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:38617 (local address 127.0.0.1:33993)
0811 17:18:38.649957 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:38.649975 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:38.650015 (+ 40us) server_negotiation.cc:407] Connection header received
0811 17:18:38.650730 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:38.650749 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:38.651056 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:38.651398 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:38.652319 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:38.652324 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:38.652337 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:38.652347 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:38.652435 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:38.652569 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:38.652690 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:38.652881 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 17:18:38.653067 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":150,"threads_started":1}
I20260811 17:18:38.654886 14338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3: No bootstrap required, opened a new log
I20260811 17:18:38.657382 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:38.659240 14338 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c222c252e04cc1ba7f4b7a7e8d40a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33993 } }
I20260811 17:18:38.659611 14338 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:38.659742 14338 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5c222c252e04cc1ba7f4b7a7e8d40a3, State: Initialized, Role: FOLLOWER
I20260811 17:18:38.660331 14338 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c222c252e04cc1ba7f4b7a7e8d40a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33993 } }
I20260811 17:18:38.660490 14338 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:38.660589 14338 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:38.660743 14338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:38.661687 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:38.659600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:49227 (local address 127.0.0.1:33993)
0811 17:18:38.659747 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:38.659754 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:38.659938 (+ 184us) server_negotiation.cc:407] Connection header received
0811 17:18:38.660111 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:38.660114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:38.660157 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:38.660246 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:38.661205 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:38.661208 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:38.661211 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:38.661223 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:38.661253 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:38.661265 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:38.661359 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:38.661520 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 17:18:38.661572 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:18:38.662132 14338 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c222c252e04cc1ba7f4b7a7e8d40a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33993 } }
I20260811 17:18:38.662535 14338 leader_election.cc:302] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5c222c252e04cc1ba7f4b7a7e8d40a3; no voters:
I20260811 17:18:38.662887 14338 leader_election.cc:288] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:38.663030 14350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:38.663313 14350 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3 [term 1 LEADER]: Becoming Leader. State: Replica: b5c222c252e04cc1ba7f4b7a7e8d40a3, State: Running, Role: LEADER
I20260811 17:18:38.663705 14350 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c222c252e04cc1ba7f4b7a7e8d40a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33993 } }
I20260811 17:18:38.664256 14338 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:38.665616 14353 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5c222c252e04cc1ba7f4b7a7e8d40a3. Latest consensus state: current_term: 1 leader_uuid: "b5c222c252e04cc1ba7f4b7a7e8d40a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c222c252e04cc1ba7f4b7a7e8d40a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33993 } } }
I20260811 17:18:38.665760 14353 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:38.666167 14361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:38.666149 14351 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5c222c252e04cc1ba7f4b7a7e8d40a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5c222c252e04cc1ba7f4b7a7e8d40a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33993 } } }
I20260811 17:18:38.666306 14351 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:38.670469 14361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:38.678421 14361 catalog_manager.cc:1347] Generated new cluster ID: 2c69a82c73a04285b073adfac14e0c97
I20260811 17:18:38.678584 14361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:38.701295 14361 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:38.701668 14361 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:38.717226 14361 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5c222c252e04cc1ba7f4b7a7e8d40a3: Generated new TSK 0
I20260811 17:18:38.718258 14361 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:38.727051 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 14261
2026-08-11T17:18:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1557 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:38 dist-test-slave-fd7l krb5kdc[14375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:38 dist-test-slave-fd7l krb5kdc[14375](info): set up 2 sockets
Aug 11 17:18:38 dist-test-slave-fd7l krb5kdc[14375](info): commencing operation
krb5kdc: starting...
W20260811 17:18:39.859401 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:39 dist-test-slave-fd7l krb5kdc[14375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:18:40.004101 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:46343
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:39269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:46343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:40.121780 14390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:40.122119 14390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:40.122255 14390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:40.126368 14390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:40.126436 14390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:40.126457 14390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:40.126474 14390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:40.126489 14390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:40.132076 14390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:39269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:46343
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:46343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:40.133250 14390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:40.134281 14390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:40.143400 14395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:40.143465 14390 server_base.cc:1034] running on GCE node
W20260811 17:18:40.143421 14396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:40.143425 14398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:40.144327 14390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:40.145592 14390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:40.146770 14390 hybrid_clock.cc:648] HybridClock initialized: now 1786468720146754 us; error 28 us; skew 500 ppm
Aug 11 17:18:40 dist-test-slave-fd7l krb5kdc[14375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468720, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:40.149935 14390 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:18:40.150955 14390 webserver.cc:460] Webserver started at http://127.21.54.190:34211/ using document root <none> and password file <none>
I20260811 17:18:40.151525 14390 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:40.151623 14390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:40.151880 14390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:40.154129 14390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "e01c783276914dd99deefabe897bb2a7"
format_stamp: "Formatted at 2026-08-11 17:18:40 on dist-test-slave-fd7l"
server_key: "d3f8e10f4a9150f540fdae1892ced795"
server_key_iv: "8c31e86e4942ca102d48fd9caff0e6ca"
server_key_version: "encryptionkey@0"
I20260811 17:18:40.154747 14390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "e01c783276914dd99deefabe897bb2a7"
format_stamp: "Formatted at 2026-08-11 17:18:40 on dist-test-slave-fd7l"
server_key: "d3f8e10f4a9150f540fdae1892ced795"
server_key_iv: "8c31e86e4942ca102d48fd9caff0e6ca"
server_key_version: "encryptionkey@0"
I20260811 17:18:40.158914 14390 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:18:40.161538 14406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:40.163127 14390 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:40.163379 14390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "e01c783276914dd99deefabe897bb2a7"
format_stamp: "Formatted at 2026-08-11 17:18:40 on dist-test-slave-fd7l"
server_key: "d3f8e10f4a9150f540fdae1892ced795"
server_key_iv: "8c31e86e4942ca102d48fd9caff0e6ca"
server_key_version: "encryptionkey@0"
I20260811 17:18:40.163592 14390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:40.188190 14390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:40.192190 14390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:40.192552 14390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:40.200881 14390 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:46343
I20260811 17:18:40.200912 14467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:46343 every 8 connection(s)
I20260811 17:18:40.202049 14390 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:40.202718 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 14390
I20260811 17:18:40.202926 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:40.203254 21722 external_mini_cluster.cc:1428] Setting key f9d2cb2560bb7adf6ad78432b8e4fdbf
I20260811 17:18:40.205775 14468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:40.213578 14468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7: Bootstrap starting.
Aug 11 17:18:40 dist-test-slave-fd7l krb5kdc[14375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:18:40.216881 14468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:40.217974 14468 log.cc:824] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:40.220795 14468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7: No bootstrap required, opened a new log
I20260811 17:18:40.221650 14471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:40.204971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54145 (local address 127.21.54.190:46343)
0811 17:18:40.205478 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:40.205514 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:40.205563 (+ 49us) server_negotiation.cc:407] Connection header received
0811 17:18:40.206517 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:40.206543 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:40.207006 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:40.207413 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:40.208211 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:40.209999 (+ 1788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:40.211029 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:40.211834 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:40.212020 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:40.215286 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:40.215294 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:40.215316 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:40.215357 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:40.217889 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:40.218582 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:40.218589 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:40.218594 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:40.218725 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:40.219185 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:40.219189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:40.219191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:40.219787 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:40.220025 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:40.220323 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 17:18:40.220617 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":139,"threads_started":1}
I20260811 17:18:40.225274 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:40.225319 14468 raft_consensus.cc:348] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e01c783276914dd99deefabe897bb2a7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46343 } }
I20260811 17:18:40.225605 14468 raft_consensus.cc:374] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:40.225660 14468 raft_consensus.cc:729] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01c783276914dd99deefabe897bb2a7, State: Initialized, Role: FOLLOWER
I20260811 17:18:40.226181 14468 consensus_queue.cc:260] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e01c783276914dd99deefabe897bb2a7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46343 } }
I20260811 17:18:40.226367 14468 raft_consensus.cc:388] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:40.226436 14468 raft_consensus.cc:482] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:40.226543 14468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:40.227885 14468 raft_consensus.cc:504] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e01c783276914dd99deefabe897bb2a7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46343 } }
I20260811 17:18:40.228303 14468 leader_election.cc:302] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e01c783276914dd99deefabe897bb2a7; no voters:
I20260811 17:18:40.228766 14468 leader_election.cc:288] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:40.228879 14479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:40.229169 14479 raft_consensus.cc:686] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7 [term 1 LEADER]: Becoming Leader. State: Replica: e01c783276914dd99deefabe897bb2a7, State: Running, Role: LEADER
I20260811 17:18:40.229585 14479 consensus_queue.cc:237] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e01c783276914dd99deefabe897bb2a7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46343 } }
I20260811 17:18:40.230237 14468 sys_catalog.cc:564] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:40.231529 14481 sys_catalog.cc:455] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e01c783276914dd99deefabe897bb2a7. Latest consensus state: current_term: 1 leader_uuid: "e01c783276914dd99deefabe897bb2a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e01c783276914dd99deefabe897bb2a7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46343 } } }
I20260811 17:18:40.231529 14480 sys_catalog.cc:455] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e01c783276914dd99deefabe897bb2a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e01c783276914dd99deefabe897bb2a7" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46343 } } }
I20260811 17:18:40.231706 14480 sys_catalog.cc:458] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:40.231706 14481 sys_catalog.cc:458] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7 [sys.catalog]: This master's current role is: LEADER
W20260811 17:18:40.235723 14497 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:18:40.235841 14497 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:18:40.235985 14491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:40.237061 14491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:40.238164 14471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:40.227730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44411 (local address 127.21.54.190:46343)
0811 17:18:40.227886 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:40.227891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:40.228096 (+ 205us) server_negotiation.cc:407] Connection header received
0811 17:18:40.228235 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:40.228238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:40.228281 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:40.228353 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:40.229262 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:40.230011 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:40.231250 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:40.231620 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:40.231697 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:40.234047 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:40.234051 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:40.234057 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:40.234088 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:40.235952 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:40.236594 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:40.236598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:40.236600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:40.236702 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:40.237117 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:40.237122 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:40.237124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:40.237370 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:40.237536 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:40.237893 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 17:18:40.238006 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 17:18:40.243049 14491 catalog_manager.cc:1347] Generated new cluster ID: b2f87e4c9a6b42009f5f03ac515d3a71
I20260811 17:18:40.243130 14491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:40.263406 14491 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:40.264819 14491 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:40.271221 14491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e01c783276914dd99deefabe897bb2a7: Generated new TSK 0
I20260811 17:18:40.272109 14491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:40.276790 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 14390
W20260811 17:18:40.286722 14373 connection.cc:535] client connection to 127.21.54.190:46343 recv error: Network error: failed to read from TLS socket (remote: 127.21.54.190:46343): Connection reset by peer (error 104)
2026-08-11T17:18:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1549 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:40 dist-test-slave-fd7l krb5kdc[14503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:40 dist-test-slave-fd7l krb5kdc[14503](info): set up 2 sockets
Aug 11 17:18:40 dist-test-slave-fd7l krb5kdc[14503](info): commencing operation
krb5kdc: starting...
W20260811 17:18:41.418798 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:41 dist-test-slave-fd7l krb5kdc[14503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:18:41.559996 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:42825
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:33051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:42825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:41.682237 14518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:41.682545 14518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:41.682591 14518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:41.686887 14518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:41.686966 14518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:41.686987 14518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:41.687005 14518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:41.687022 14518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:41.692682 14518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:33051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:42825
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:42825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:41.693903 14518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:41.695010 14518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:41.704093 14527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:41.704195 14518 server_base.cc:1034] running on GCE node
W20260811 17:18:41.704066 14529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:41.704071 14526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:41.704972 14518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:41.706156 14518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:41.707331 14518 hybrid_clock.cc:648] HybridClock initialized: now 1786468721707256 us; error 88 us; skew 500 ppm
Aug 11 17:18:41 dist-test-slave-fd7l krb5kdc[14503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468721, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:41.710351 14518 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:18:41.711259 14518 webserver.cc:460] Webserver started at http://127.21.54.190:35577/ using document root <none> and password file <none>
I20260811 17:18:41.711798 14518 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:41.711858 14518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:41.712036 14518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:41.714172 14518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "b5e8953c1338401da25969475a89e1c9"
format_stamp: "Formatted at 2026-08-11 17:18:41 on dist-test-slave-fd7l"
server_key: "054573094e76d7df51e47c9f2cb8ef06"
server_key_iv: "04204e06d62a6d1cc5e0b89d135e640e"
server_key_version: "encryptionkey@0"
I20260811 17:18:41.714706 14518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "b5e8953c1338401da25969475a89e1c9"
format_stamp: "Formatted at 2026-08-11 17:18:41 on dist-test-slave-fd7l"
server_key: "054573094e76d7df51e47c9f2cb8ef06"
server_key_iv: "04204e06d62a6d1cc5e0b89d135e640e"
server_key_version: "encryptionkey@0"
I20260811 17:18:41.718614 14518 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:18:41.721166 14540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:41.722477 14518 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 17:18:41.722599 14518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "b5e8953c1338401da25969475a89e1c9"
format_stamp: "Formatted at 2026-08-11 17:18:41 on dist-test-slave-fd7l"
server_key: "054573094e76d7df51e47c9f2cb8ef06"
server_key_iv: "04204e06d62a6d1cc5e0b89d135e640e"
server_key_version: "encryptionkey@0"
I20260811 17:18:41.722708 14518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:41.742700 14518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:41.745832 14518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:41.746004 14518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:41.753871 14518 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:42825
I20260811 17:18:41.753895 14602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:42825 every 8 connection(s)
I20260811 17:18:41.754910 14518 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:41.757815 14603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:41.758673 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 14518
I20260811 17:18:41.758831 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:41.759138 21722 external_mini_cluster.cc:1428] Setting key 2f6f5923645cfdf57bce56b50692c52c
I20260811 17:18:41.763974 14603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9: Bootstrap starting.
I20260811 17:18:41.766381 14603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:41.767150 14603 log.cc:824] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:41.768496 14606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:41.760616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60497 (local address 127.21.54.190:42825)
0811 17:18:41.761076 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:41.761092 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:41.761128 (+ 36us) server_negotiation.cc:407] Connection header received
0811 17:18:41.761749 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:41.761765 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:41.762209 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:41.762598 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:41.762992 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:41.763954 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:41.764925 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:41.765409 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:41.765610 (+ 201us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:41.766584 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:41.766589 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:41.766602 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:41.766993 (+ 391us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:41.767058 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:41.767082 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:41.767227 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:41.767462 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 17:18:41.767608 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":150,"threads_started":1}
I20260811 17:18:41.769521 14603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9: No bootstrap required, opened a new log
I20260811 17:18:41.771855 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:41.773603 14603 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e8953c1338401da25969475a89e1c9" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42825 } }
I20260811 17:18:41.773896 14603 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:41.774017 14603 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5e8953c1338401da25969475a89e1c9, State: Initialized, Role: FOLLOWER
I20260811 17:18:41.774534 14603 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e8953c1338401da25969475a89e1c9" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42825 } }
I20260811 17:18:41.774677 14603 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:41.774799 14603 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:41.774926 14603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:41.775969 14603 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e8953c1338401da25969475a89e1c9" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42825 } }
I20260811 17:18:41.776355 14603 leader_election.cc:302] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5e8953c1338401da25969475a89e1c9; no voters:
I20260811 17:18:41.776758 14603 leader_election.cc:288] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:41.776912 14615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:41.777184 14615 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9 [term 1 LEADER]: Becoming Leader. State: Replica: b5e8953c1338401da25969475a89e1c9, State: Running, Role: LEADER
I20260811 17:18:41.777592 14615 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e8953c1338401da25969475a89e1c9" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42825 } }
I20260811 17:18:41.778256 14603 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:41.778935 14606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:41.774144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36375 (local address 127.21.54.190:42825)
0811 17:18:41.774258 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:41.774263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:41.774446 (+ 183us) server_negotiation.cc:407] Connection header received
0811 17:18:41.774595 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:41.774603 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:41.774667 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:41.774753 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:41.775075 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:41.775611 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:41.776612 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:41.777120 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:41.777225 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:41.778368 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:41.778372 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:41.778375 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:41.778389 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:41.778431 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:41.778444 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:41.778541 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:41.778747 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 17:18:41.778787 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 17:18:41.779759 14617 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5e8953c1338401da25969475a89e1c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e8953c1338401da25969475a89e1c9" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42825 } } }
I20260811 17:18:41.778941 14615 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5e8953c1338401da25969475a89e1c9. Latest consensus state: current_term: 1 leader_uuid: "b5e8953c1338401da25969475a89e1c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e8953c1338401da25969475a89e1c9" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 42825 } } }
I20260811 17:18:41.780489 14615 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:41.780457 14617 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9 [sys.catalog]: This master's current role is: LEADER
W20260811 17:18:41.783666 14631 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:18:41.783764 14631 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:18:41.783861 14633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:41.784834 14633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:41.790609 14633 catalog_manager.cc:1347] Generated new cluster ID: 3c903a9b2fc44604bb2f7705b40b64df
I20260811 17:18:41.790750 14633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:41.811317 14633 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:41.812853 14633 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:41.830236 14633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5e8953c1338401da25969475a89e1c9: Generated new TSK 0
I20260811 17:18:41.831230 14633 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:41.842451 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 14518
W20260811 17:18:41.853981 14501 connection.cc:535] client connection to 127.21.54.190:42825 recv error: Network error: failed to read from TLS socket (remote: 127.21.54.190:42825): Connection reset by peer (error 104)
2026-08-11T17:18:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1567 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:41 dist-test-slave-fd7l krb5kdc[14639](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:41 dist-test-slave-fd7l krb5kdc[14639](info): set up 2 sockets
Aug 11 17:18:41 dist-test-slave-fd7l krb5kdc[14639](info): commencing operation
krb5kdc: starting...
W20260811 17:18:42.974505 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:43 dist-test-slave-fd7l krb5kdc[14639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:18:43.116685 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:45985
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:40379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:45985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:43.239583 14653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:43.239952 14653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:43.240036 14653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:43.244272 14653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:43.244388 14653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:43.244439 14653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:43.244500 14653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:43.244567 14653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:43.250243 14653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:40379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:45985
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:45985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:43.251534 14653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:43.252732 14653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:43.262553 14658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:43.262753 14653 server_base.cc:1034] running on GCE node
W20260811 17:18:43.262565 14659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:43.262586 14661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:43.263504 14653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:43.264911 14653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:43.266103 14653 hybrid_clock.cc:648] HybridClock initialized: now 1786468723266073 us; error 45 us; skew 500 ppm
Aug 11 17:18:43 dist-test-slave-fd7l krb5kdc[14639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468723, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:43.275982 14653 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:18:43.277275 14653 webserver.cc:460] Webserver started at http://127.21.54.190:38965/ using document root <none> and password file <none>
I20260811 17:18:43.277992 14653 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:43.278116 14653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:43.278415 14653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:43.280982 14653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "75249f6e03c34fb3b2c6c448c2b58a98"
format_stamp: "Formatted at 2026-08-11 17:18:43 on dist-test-slave-fd7l"
server_key: "50f7404da1b26b53d07cb9b2e0c543b3"
server_key_iv: "023f6861aff68b734086f01d4e3fd714"
server_key_version: "encryptionkey@0"
I20260811 17:18:43.281747 14653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "75249f6e03c34fb3b2c6c448c2b58a98"
format_stamp: "Formatted at 2026-08-11 17:18:43 on dist-test-slave-fd7l"
server_key: "50f7404da1b26b53d07cb9b2e0c543b3"
server_key_iv: "023f6861aff68b734086f01d4e3fd714"
server_key_version: "encryptionkey@0"
I20260811 17:18:43.286250 14653 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:18:43.289194 14669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:43.290709 14653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:18:43.290899 14653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "75249f6e03c34fb3b2c6c448c2b58a98"
format_stamp: "Formatted at 2026-08-11 17:18:43 on dist-test-slave-fd7l"
server_key: "50f7404da1b26b53d07cb9b2e0c543b3"
server_key_iv: "023f6861aff68b734086f01d4e3fd714"
server_key_version: "encryptionkey@0"
I20260811 17:18:43.291049 14653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:43.296550 14653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:43.300976 14653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:43.301227 14653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:43.309715 14653 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:45985
I20260811 17:18:43.309746 14731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:45985 every 8 connection(s)
I20260811 17:18:43.310808 14653 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:43.313915 14732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:43.315441 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 14653
I20260811 17:18:43.315634 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:43.315944 21722 external_mini_cluster.cc:1428] Setting key 7add6a678b984179fa569398caef6999
I20260811 17:18:43.320454 14732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98: Bootstrap starting.
I20260811 17:18:43.321576 14735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:43.317533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55079 (local address 127.21.54.190:45985)
0811 17:18:43.318023 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:43.318039 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:43.318087 (+ 48us) server_negotiation.cc:407] Connection header received
0811 17:18:43.318734 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:43.318752 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:43.319038 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:43.319443 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:43.320338 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:43.320343 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:43.320354 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:43.320365 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:43.320418 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:43.320438 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:43.320619 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:43.320692 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 17:18:43.320864 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":150,"threads_started":1}
I20260811 17:18:43.323143 14732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:43.324079 14732 log.cc:824] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:43.324635 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:43.326610 14732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98: No bootstrap required, opened a new log
I20260811 17:18:43.328825 14735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:43.327049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47675 (local address 127.21.54.190:45985)
0811 17:18:43.327167 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:43.327178 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:43.327404 (+ 226us) server_negotiation.cc:407] Connection header received
0811 17:18:43.327498 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:43.327501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:43.327538 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:43.327627 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:43.328422 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:43.328425 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:43.328428 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:43.328440 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:43.328471 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:43.328486 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:43.328600 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:43.328660 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 17:18:43.328699 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 17:18:43.330979 14732 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75249f6e03c34fb3b2c6c448c2b58a98" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45985 } }
I20260811 17:18:43.331225 14732 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:43.331295 14732 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75249f6e03c34fb3b2c6c448c2b58a98, State: Initialized, Role: FOLLOWER
I20260811 17:18:43.331759 14732 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75249f6e03c34fb3b2c6c448c2b58a98" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45985 } }
I20260811 17:18:43.331909 14732 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:43.331995 14732 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:43.332119 14732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:43.333221 14732 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75249f6e03c34fb3b2c6c448c2b58a98" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45985 } }
I20260811 17:18:43.333600 14732 leader_election.cc:302] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75249f6e03c34fb3b2c6c448c2b58a98; no voters:
I20260811 17:18:43.333954 14732 leader_election.cc:288] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:43.334197 14742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:43.334738 14742 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98 [term 1 LEADER]: Becoming Leader. State: Replica: 75249f6e03c34fb3b2c6c448c2b58a98, State: Running, Role: LEADER
I20260811 17:18:43.335224 14742 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75249f6e03c34fb3b2c6c448c2b58a98" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45985 } }
I20260811 17:18:43.335507 14732 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:43.337304 14743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75249f6e03c34fb3b2c6c448c2b58a98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75249f6e03c34fb3b2c6c448c2b58a98" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45985 } } }
I20260811 17:18:43.337361 14744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75249f6e03c34fb3b2c6c448c2b58a98. Latest consensus state: current_term: 1 leader_uuid: "75249f6e03c34fb3b2c6c448c2b58a98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75249f6e03c34fb3b2c6c448c2b58a98" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45985 } } }
I20260811 17:18:43.337462 14743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:43.337476 14744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:43.337852 14758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:43.341787 14758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:43.348006 14758 catalog_manager.cc:1347] Generated new cluster ID: 2ad3f824a66e4ac89920fa12112d3848
I20260811 17:18:43.348114 14758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:43.364455 14758 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:43.364904 14758 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:43.375002 14758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75249f6e03c34fb3b2c6c448c2b58a98: Generated new TSK 0
I20260811 17:18:43.375891 14758 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:43.386840 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 14653
2026-08-11T17:18:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1542 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:43 dist-test-slave-fd7l krb5kdc[14766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:43 dist-test-slave-fd7l krb5kdc[14766](info): set up 2 sockets
Aug 11 17:18:43 dist-test-slave-fd7l krb5kdc[14766](info): commencing operation
krb5kdc: starting...
W20260811 17:18:44.523124 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:44 dist-test-slave-fd7l krb5kdc[14766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:18:44.666288 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:44931
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:41709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:44931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:44.786254 14780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:44.786636 14780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:44.786716 14780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:44.791003 14780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:44.791121 14780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:44.791163 14780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:44.791201 14780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:44.791234 14780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:44.796794 14780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:41709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:44931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:44931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:44.798003 14780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:44.799108 14780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:44.808328 14786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:44.808346 14785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:44.808383 14790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:44.808558 14780 server_base.cc:1034] running on GCE node
I20260811 17:18:44.809278 14780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:44.810382 14780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:44.811609 14780 hybrid_clock.cc:648] HybridClock initialized: now 1786468724811576 us; error 45 us; skew 500 ppm
Aug 11 17:18:44 dist-test-slave-fd7l krb5kdc[14766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468724, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:44.814684 14780 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:18:44.815636 14780 webserver.cc:460] Webserver started at http://127.21.54.190:46651/ using document root <none> and password file <none>
I20260811 17:18:44.816287 14780 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:44.816401 14780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:44.816730 14780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:44.819418 14780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "1ff0387856bb46118fc531e20a2dc2de"
format_stamp: "Formatted at 2026-08-11 17:18:44 on dist-test-slave-fd7l"
server_key: "dfdfbfaf565b180c9b9b964719a30dd4"
server_key_iv: "806736984512b609e47bc26901682fb1"
server_key_version: "encryptionkey@0"
I20260811 17:18:44.820107 14780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "1ff0387856bb46118fc531e20a2dc2de"
format_stamp: "Formatted at 2026-08-11 17:18:44 on dist-test-slave-fd7l"
server_key: "dfdfbfaf565b180c9b9b964719a30dd4"
server_key_iv: "806736984512b609e47bc26901682fb1"
server_key_version: "encryptionkey@0"
I20260811 17:18:44.824160 14780 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:18:44.827064 14798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:44.828325 14780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:18:44.828485 14780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "1ff0387856bb46118fc531e20a2dc2de"
format_stamp: "Formatted at 2026-08-11 17:18:44 on dist-test-slave-fd7l"
server_key: "dfdfbfaf565b180c9b9b964719a30dd4"
server_key_iv: "806736984512b609e47bc26901682fb1"
server_key_version: "encryptionkey@0"
I20260811 17:18:44.828737 14780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:44.856755 14780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:44.860371 14780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:44.860648 14780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:44.868572 14780 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:44931
I20260811 17:18:44.868602 14859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:44931 every 8 connection(s)
I20260811 17:18:44.869624 14780 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:44.872538 14861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:44.874997 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 14780
I20260811 17:18:44.875166 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:44.875499 21722 external_mini_cluster.cc:1428] Setting key f5f595857c713226b1b1bc6d338927fe
I20260811 17:18:44.879297 14861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de: Bootstrap starting.
I20260811 17:18:44.881857 14861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:44.882671 14861 log.cc:824] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de: Log is configured to *not* fsync() on all Append() calls
Aug 11 17:18:44 dist-test-slave-fd7l krb5kdc[14766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:18:44.885222 14861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de: No bootstrap required, opened a new log
I20260811 17:18:44.888732 14861 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff0387856bb46118fc531e20a2dc2de" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44931 } }
I20260811 17:18:44.889003 14861 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:44.889088 14861 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ff0387856bb46118fc531e20a2dc2de, State: Initialized, Role: FOLLOWER
I20260811 17:18:44.889657 14861 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff0387856bb46118fc531e20a2dc2de" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44931 } }
I20260811 17:18:44.889854 14861 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:44.889930 14861 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:44.890044 14861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:44.891387 14861 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff0387856bb46118fc531e20a2dc2de" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44931 } }
I20260811 17:18:44.891788 14861 leader_election.cc:302] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ff0387856bb46118fc531e20a2dc2de; no voters:
I20260811 17:18:44.892156 14861 leader_election.cc:288] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:44.892287 14867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:44.892562 14867 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de [term 1 LEADER]: Becoming Leader. State: Replica: 1ff0387856bb46118fc531e20a2dc2de, State: Running, Role: LEADER
I20260811 17:18:44.892663 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:44.877067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59901 (local address 127.21.54.190:44931)
0811 17:18:44.877559 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:44.877577 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:44.877615 (+ 38us) server_negotiation.cc:407] Connection header received
0811 17:18:44.878276 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:44.878294 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:44.878672 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:44.879039 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:44.879899 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:44.881012 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:44.881984 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:44.882496 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:44.882682 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:44.886054 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:44.886060 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:44.886076 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:44.886110 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:44.888703 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:44.889280 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:44.889286 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:44.889290 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:44.889440 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:44.889785 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:44.889787 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:44.889789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:44.890369 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:44.890601 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:44.890951 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 17:18:44.891443 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":151,"threads_started":1}
I20260811 17:18:44.892975 14867 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff0387856bb46118fc531e20a2dc2de" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44931 } }
I20260811 17:18:44.893656 14861 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:44.894887 14871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ff0387856bb46118fc531e20a2dc2de. Latest consensus state: current_term: 1 leader_uuid: "1ff0387856bb46118fc531e20a2dc2de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff0387856bb46118fc531e20a2dc2de" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44931 } } }
I20260811 17:18:44.895325 14871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:44.895211 14868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ff0387856bb46118fc531e20a2dc2de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff0387856bb46118fc531e20a2dc2de" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 44931 } } }
I20260811 17:18:44.895710 14868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:44.896163 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:44.896224 14885 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:44.900472 14885 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:44.906987 14885 catalog_manager.cc:1347] Generated new cluster ID: ee724a4aa3e94aae819ab0cefa33161c
I20260811 17:18:44.907166 14885 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:44.907397 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:44.898584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57779 (local address 127.21.54.190:44931)
0811 17:18:44.898702 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:44.898706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:44.898995 (+ 289us) server_negotiation.cc:407] Connection header received
0811 17:18:44.899160 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:44.899163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:44.899205 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:44.899292 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:44.900038 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:44.900683 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:44.901489 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:44.901736 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:44.901850 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:44.903532 (+ 1682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:44.903536 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:44.903540 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:44.903570 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:44.905403 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:44.905914 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:44.905918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:44.905920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:44.906014 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:44.906326 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:44.906329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:44.906332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:44.906582 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:44.906716 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:44.907095 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 17:18:44.907232 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 17:18:44.923849 14885 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:44.925303 14885 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:44.933569 14885 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ff0387856bb46118fc531e20a2dc2de: Generated new TSK 0
I20260811 17:18:44.934367 14885 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:44.942098 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 14780
2026-08-11T17:18:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1555 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:45 dist-test-slave-fd7l krb5kdc[14902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:45 dist-test-slave-fd7l krb5kdc[14902](info): set up 2 sockets
Aug 11 17:18:45 dist-test-slave-fd7l krb5kdc[14902](info): commencing operation
krb5kdc: starting...
W20260811 17:18:46.119037 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:46 dist-test-slave-fd7l krb5kdc[14902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:18:46.263585 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:45815
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:43063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:45815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:46.381733 14916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:46.382076 14916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:46.382180 14916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:46.386420 14916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:46.386540 14916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:46.386582 14916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:46.386652 14916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:46.386691 14916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:46.392381 14916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:43063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:45815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:45815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:46.393678 14916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:46.394815 14916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:46.404242 14921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:46.404242 14922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:46.404265 14924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:46.404366 14916 server_base.cc:1034] running on GCE node
I20260811 17:18:46.405308 14916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:46.406515 14916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:46.407735 14916 hybrid_clock.cc:648] HybridClock initialized: now 1786468726407722 us; error 31 us; skew 500 ppm
Aug 11 17:18:46 dist-test-slave-fd7l krb5kdc[14902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468726, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:46.410880 14916 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:18:46.411880 14916 webserver.cc:460] Webserver started at http://127.21.54.190:32849/ using document root <none> and password file <none>
I20260811 17:18:46.412582 14916 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:46.412691 14916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:46.412962 14916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:46.415210 14916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "20e01cf05a98456f91968d9403fc0317"
format_stamp: "Formatted at 2026-08-11 17:18:46 on dist-test-slave-fd7l"
server_key: "c266b127e11c5b1e4ccb285ebecf594e"
server_key_iv: "14f6faddd9b14c979c2b418f0afe3470"
server_key_version: "encryptionkey@0"
I20260811 17:18:46.415840 14916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "20e01cf05a98456f91968d9403fc0317"
format_stamp: "Formatted at 2026-08-11 17:18:46 on dist-test-slave-fd7l"
server_key: "c266b127e11c5b1e4ccb285ebecf594e"
server_key_iv: "14f6faddd9b14c979c2b418f0afe3470"
server_key_version: "encryptionkey@0"
I20260811 17:18:46.419868 14916 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:18:46.423002 14931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:46.424392 14916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:18:46.424583 14916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "20e01cf05a98456f91968d9403fc0317"
format_stamp: "Formatted at 2026-08-11 17:18:46 on dist-test-slave-fd7l"
server_key: "c266b127e11c5b1e4ccb285ebecf594e"
server_key_iv: "14f6faddd9b14c979c2b418f0afe3470"
server_key_version: "encryptionkey@0"
I20260811 17:18:46.424719 14916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:46.449688 14916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:46.453493 14916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:46.453780 14916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:46.461509 14916 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:45815
I20260811 17:18:46.461536 14995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:45815 every 8 connection(s)
I20260811 17:18:46.462673 14916 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:46.465616 14996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:46.471778 14996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317: Bootstrap starting.
I20260811 17:18:46.472244 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 14916
I20260811 17:18:46.472427 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:46.472757 21722 external_mini_cluster.cc:1428] Setting key e84c9b0dcb36713466e1027494e57364
I20260811 17:18:46.474792 14996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:46.475584 14996 log.cc:824] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:46.477761 14996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317: No bootstrap required, opened a new log
I20260811 17:18:46.482303 14996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e01cf05a98456f91968d9403fc0317" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45815 } }
I20260811 17:18:46.482583 14996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:46.482465 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:46.474285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39601 (local address 127.21.54.190:45815)
0811 17:18:46.474727 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:46.474748 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:46.474800 (+ 52us) server_negotiation.cc:407] Connection header received
0811 17:18:46.475514 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:46.475531 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:46.475959 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:46.476341 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:46.476773 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:46.477746 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:46.479001 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:46.479479 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:46.479609 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:46.480471 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:46.480477 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:46.480490 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:46.481014 (+ 524us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:46.481071 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:46.481095 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:46.481176 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:46.481389 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 17:18:46.481685 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":170,"threads_started":1}
I20260811 17:18:46.482671 14996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20e01cf05a98456f91968d9403fc0317, State: Initialized, Role: FOLLOWER
I20260811 17:18:46.483225 14996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e01cf05a98456f91968d9403fc0317" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45815 } }
I20260811 17:18:46.483400 14996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:46.483467 14996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:46.483564 14996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:46.484668 14996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e01cf05a98456f91968d9403fc0317" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45815 } }
I20260811 17:18:46.485049 14996 leader_election.cc:302] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20e01cf05a98456f91968d9403fc0317; no voters:
I20260811 17:18:46.485432 14996 leader_election.cc:288] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:46.485546 15001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:46.485751 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:46.485811 15001 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317 [term 1 LEADER]: Becoming Leader. State: Replica: 20e01cf05a98456f91968d9403fc0317, State: Running, Role: LEADER
I20260811 17:18:46.486137 15001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e01cf05a98456f91968d9403fc0317" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45815 } }
I20260811 17:18:46.486658 14996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:46.487412 15001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20e01cf05a98456f91968d9403fc0317. Latest consensus state: current_term: 1 leader_uuid: "20e01cf05a98456f91968d9403fc0317" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e01cf05a98456f91968d9403fc0317" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45815 } } }
I20260811 17:18:46.487524 15001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:46.487890 15002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20e01cf05a98456f91968d9403fc0317" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e01cf05a98456f91968d9403fc0317" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 45815 } } }
I20260811 17:18:46.488021 15002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317 [sys.catalog]: This master's current role is: LEADER
W20260811 17:18:46.491849 15026 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:18:46.491942 15026 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:18:46.492002 15018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:46.492962 15018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:46.494680 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:46.488491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51673 (local address 127.21.54.190:45815)
0811 17:18:46.488622 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:46.488627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:46.489763 (+ 1136us) server_negotiation.cc:407] Connection header received
0811 17:18:46.489908 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:46.489911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:46.489961 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:46.490053 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:46.490402 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:46.491174 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:46.492315 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:46.492675 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:46.492788 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:46.493870 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:46.493874 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:46.493877 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:46.493892 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:46.493921 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:46.493935 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:46.494002 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:46.494467 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 17:18:46.494545 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 17:18:46.498746 15018 catalog_manager.cc:1347] Generated new cluster ID: f944bbe811f445a281711ecad3b97a55
I20260811 17:18:46.498827 15018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:46.511639 15018 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:46.513024 15018 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:46.522053 15018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20e01cf05a98456f91968d9403fc0317: Generated new TSK 0
I20260811 17:18:46.522837 15018 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:46.531852 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 14916
2026-08-11T17:18:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1591 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:46 dist-test-slave-fd7l krb5kdc[15032](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:46 dist-test-slave-fd7l krb5kdc[15032](info): set up 2 sockets
Aug 11 17:18:46 dist-test-slave-fd7l krb5kdc[15032](info): commencing operation
krb5kdc: starting...
W20260811 17:18:47.674701 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:47 dist-test-slave-fd7l krb5kdc[15032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:18:47.817034 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:46237
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:33615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:46237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:47.939807 15048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:47.940183 15048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:47.940263 15048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:47.944676 15048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:47.944788 15048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:47.944824 15048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:47.944860 15048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:47.944897 15048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:47.950922 15048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:33615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:46237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:46237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:47.952214 15048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:47.953426 15048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:47.962996 15056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:47.962996 15054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:47.963034 15053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:47.963593 15048 server_base.cc:1034] running on GCE node
I20260811 17:18:47.964231 15048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:47.965514 15048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:47.966727 15048 hybrid_clock.cc:648] HybridClock initialized: now 1786468727966703 us; error 44 us; skew 500 ppm
Aug 11 17:18:47 dist-test-slave-fd7l krb5kdc[15032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468727, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:47.970008 15048 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:18:47.971061 15048 webserver.cc:460] Webserver started at http://127.21.54.190:38777/ using document root <none> and password file <none>
I20260811 17:18:47.971652 15048 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:47.971715 15048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:47.971897 15048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:47.974201 15048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "57662794cf104ab58e3a372831f56bd4"
format_stamp: "Formatted at 2026-08-11 17:18:47 on dist-test-slave-fd7l"
server_key: "54fdaf51169daaaf2c28c7796f80c38b"
server_key_iv: "71bf06675c807b1988389b79db117e39"
server_key_version: "encryptionkey@0"
I20260811 17:18:47.974859 15048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "57662794cf104ab58e3a372831f56bd4"
format_stamp: "Formatted at 2026-08-11 17:18:47 on dist-test-slave-fd7l"
server_key: "54fdaf51169daaaf2c28c7796f80c38b"
server_key_iv: "71bf06675c807b1988389b79db117e39"
server_key_version: "encryptionkey@0"
I20260811 17:18:47.979413 15048 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:18:47.982414 15064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:47.984005 15048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:18:47.984174 15048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "57662794cf104ab58e3a372831f56bd4"
format_stamp: "Formatted at 2026-08-11 17:18:47 on dist-test-slave-fd7l"
server_key: "54fdaf51169daaaf2c28c7796f80c38b"
server_key_iv: "71bf06675c807b1988389b79db117e39"
server_key_version: "encryptionkey@0"
I20260811 17:18:47.984313 15048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:47.989862 15048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:47.993587 15048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:47.993817 15048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:48.002190 15048 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:46237
I20260811 17:18:48.002209 15131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:46237 every 8 connection(s)
I20260811 17:18:48.003345 15048 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:48.005362 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 15048
I20260811 17:18:48.005542 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:48.005906 21722 external_mini_cluster.cc:1428] Setting key 7ed7857b3cb780850602ed5345aae9a1
I20260811 17:18:48.006960 15132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:48.012043 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:48.007526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44497 (local address 127.21.54.190:46237)
0811 17:18:48.008035 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:48.008053 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:48.008103 (+ 50us) server_negotiation.cc:407] Connection header received
0811 17:18:48.009050 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:48.009074 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:48.009408 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:48.009795 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:48.010721 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:48.010726 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:48.010740 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:48.010752 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:48.010814 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:48.010836 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:48.011004 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:48.011084 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 17:18:48.011260 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":179,"threads_started":1}
I20260811 17:18:48.013784 15132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4: Bootstrap starting.
I20260811 17:18:48.015347 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:48.016454 15132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:48.017417 15132 log.cc:824] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:48.019523 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:48.017668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50251 (local address 127.21.54.190:46237)
0811 17:18:48.017807 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:48.017811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:48.018029 (+ 218us) server_negotiation.cc:407] Connection header received
0811 17:18:48.018194 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:48.018197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:48.018237 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:48.018315 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:48.019072 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:48.019076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:48.019079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:18:48.019090 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:48.019119 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:18:48.019137 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:48.019223 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:48.019304 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 17:18:48.019343 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 17:18:48.019699 15132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4: No bootstrap required, opened a new log
I20260811 17:18:48.023667 15132 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57662794cf104ab58e3a372831f56bd4" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46237 } }
I20260811 17:18:48.023952 15132 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:18:48.023996 15132 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57662794cf104ab58e3a372831f56bd4, State: Initialized, Role: FOLLOWER
I20260811 17:18:48.024461 15132 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57662794cf104ab58e3a372831f56bd4" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46237 } }
I20260811 17:18:48.024667 15132 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:18:48.024721 15132 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:18:48.024817 15132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:48.025864 15132 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57662794cf104ab58e3a372831f56bd4" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46237 } }
I20260811 17:18:48.026283 15132 leader_election.cc:302] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57662794cf104ab58e3a372831f56bd4; no voters:
I20260811 17:18:48.026681 15132 leader_election.cc:288] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:18:48.026839 15145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:48.027103 15145 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4 [term 1 LEADER]: Becoming Leader. State: Replica: 57662794cf104ab58e3a372831f56bd4, State: Running, Role: LEADER
I20260811 17:18:48.027424 15145 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57662794cf104ab58e3a372831f56bd4" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46237 } }
I20260811 17:18:48.028093 15132 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:48.029277 15147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57662794cf104ab58e3a372831f56bd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57662794cf104ab58e3a372831f56bd4" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46237 } } }
I20260811 17:18:48.029433 15147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:48.029738 15148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57662794cf104ab58e3a372831f56bd4. Latest consensus state: current_term: 1 leader_uuid: "57662794cf104ab58e3a372831f56bd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57662794cf104ab58e3a372831f56bd4" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 46237 } } }
I20260811 17:18:48.029831 15148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4 [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:48.029844 15155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:48.033361 15155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:48.039687 15155 catalog_manager.cc:1347] Generated new cluster ID: 64393441757a48b882835b77650e931d
I20260811 17:18:48.039784 15155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:48.064265 15155 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:48.064728 15155 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:48.079124 15155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57662794cf104ab58e3a372831f56bd4: Generated new TSK 0
I20260811 17:18:48.080166 15155 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:18:48.090858 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 15048
2026-08-11T17:18:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1556 ms)
[----------] 18 tests from AuthTokenIssuingTest (28398 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:48 dist-test-slave-fd7l krb5kdc[15170](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:48 dist-test-slave-fd7l krb5kdc[15170](info): set up 2 sockets
Aug 11 17:18:48 dist-test-slave-fd7l krb5kdc[15170](info): commencing operation
krb5kdc: starting...
W20260811 17:18:49.305809 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:49 dist-test-slave-fd7l krb5kdc[15170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:18:49.443811 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:37271
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:37271,127.21.54.189:34149,127.21.54.188:33155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:49.567353 15188 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 17:18:49.567728 15188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:49.567802 15188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:49.567878 15188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:49.572154 15188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:49.572250 15188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:49.572289 15188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:49.572324 15188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:49.572360 15188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:49.578061 15188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:37271,127.21.54.189:34149,127.21.54.188:33155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:37271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:49.579275 15188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:49.580353 15188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:49.589524 15197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:49.589555 15195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:49.589524 15194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:49.589938 15188 server_base.cc:1034] running on GCE node
I20260811 17:18:49.590571 15188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:49.591853 15188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:49.593034 15188 hybrid_clock.cc:648] HybridClock initialized: now 1786468729593013 us; error 32 us; skew 500 ppm
Aug 11 17:18:49 dist-test-slave-fd7l krb5kdc[15170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468729, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:49.596108 15188 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:18:49.597184 15188 webserver.cc:460] Webserver started at http://127.21.54.190:39197/ using document root <none> and password file <none>
I20260811 17:18:49.597864 15188 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:49.597967 15188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:49.598232 15188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:49.600622 15188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "9a9f5740d9df4eab88769b1d708e050f"
format_stamp: "Formatted at 2026-08-11 17:18:49 on dist-test-slave-fd7l"
server_key: "e89b5aad3d5fd08e84b86a35f0454114"
server_key_iv: "c2406e88c99c9aed8d928738fd12f820"
server_key_version: "encryptionkey@0"
I20260811 17:18:49.601258 15188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "9a9f5740d9df4eab88769b1d708e050f"
format_stamp: "Formatted at 2026-08-11 17:18:49 on dist-test-slave-fd7l"
server_key: "e89b5aad3d5fd08e84b86a35f0454114"
server_key_iv: "c2406e88c99c9aed8d928738fd12f820"
server_key_version: "encryptionkey@0"
I20260811 17:18:49.605320 15188 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:18:49.607975 15206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:49.609405 15188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:49.609597 15188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "9a9f5740d9df4eab88769b1d708e050f"
format_stamp: "Formatted at 2026-08-11 17:18:49 on dist-test-slave-fd7l"
server_key: "e89b5aad3d5fd08e84b86a35f0454114"
server_key_iv: "c2406e88c99c9aed8d928738fd12f820"
server_key_version: "encryptionkey@0"
I20260811 17:18:49.609733 15188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:49.634665 15188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:49.638538 15188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:49.638777 15188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:49.646837 15188 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:37271
I20260811 17:18:49.647151 15264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:37271 every 8 connection(s)
I20260811 17:18:49.648006 15188 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:49.650928 15265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:49.651942 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 15188
I20260811 17:18:49.652127 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:49.652442 21722 external_mini_cluster.cc:1428] Setting key c2b170871775faa4ae92401fda6f6b3e
I20260811 17:18:49.654877 15265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.54.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.189@KRBTEST.COM" created.
Aug 11 17:18:49 dist-test-slave-fd7l krb5kdc[15170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468729, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:18:49.671783 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.657901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:37271 (local address 127.0.0.1:38960)
0811 17:18:49.658553 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:49.658595 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:49.659547 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:49.660287 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:49.660292 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:49.660837 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:49.661106 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:49.661122 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:49.662382 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:49.662386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:49.663199 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:49.663214 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:49.663980 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:49.663984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:49.664100 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:49.664856 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:49.664873 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:49.666608 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:49.668672 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:49.668682 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:49.668694 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:49.668927 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:49.669252 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:49.669257 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:49.669259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:49.669372 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:49.669982 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:49.669988 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:49.670134 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 17:18:49.670301 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 17:18:49.670727 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":107,"thread_start_us":169,"threads_started":1}
I20260811 17:18:49.671783 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.657900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38960 (local address 127.21.54.190:37271)
0811 17:18:49.658393 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:49.658416 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:49.658729 (+ 313us) server_negotiation.cc:407] Connection header received
0811 17:18:49.659880 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:49.659884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:49.660078 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:49.660291 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:49.661273 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:49.662237 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:49.663363 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:49.663804 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:49.663943 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:49.666773 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:49.666779 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:49.666798 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:49.666819 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:49.668546 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:49.669042 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:49.669048 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:49.669050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:49.669159 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:49.669467 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:49.669470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:49.669472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:49.669864 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:49.670017 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:49.670376 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 17:18:49.670676 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":322,"thread_start_us":152,"threads_started":1}
I20260811 17:18:49.674268 15265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:18:49.675256 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.674917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:34149 (local address 127.0.0.1:43184)
0811 17:18:49.675076 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:49.675182 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:34149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260811 17:18:49.675530 15210 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.54.189:34149: connect: Connection refused (error 111)
I20260811 17:18:49.676445 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.676197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:34149 (local address 127.0.0.1:43200)
0811 17:18:49.676309 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:49.676355 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:34149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 17:18:49.676704 15265 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.189:34149: Network error: Client connection negotiation failed: client connection to 127.21.54.189:34149: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.189.keytab.
Entry for principal kudu/127.21.54.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.189.keytab.
I20260811 17:18:49.683573 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.189:34149
--webserver_interface=127.21.54.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.189.keytab
--principal=kudu/127.21.54.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:37271,127.21.54.189:34149,127.21.54.188:33155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:18:49.725991 15265 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } attempt: 1
I20260811 17:18:49.726989 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.726642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:34149 (local address 127.0.0.1:43202)
0811 17:18:49.726834 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:49.726906 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:34149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 17:18:49.727787 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.727532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:34149 (local address 127.0.0.1:43216)
0811 17:18:49.727682 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:49.727727 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:34149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260811 17:18:49.727916 15265 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.189:34149: Network error: Client connection negotiation failed: client connection to 127.21.54.189:34149: connect: Connection refused (error 111)
I20260811 17:18:49.796093 15265 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } attempt: 2
I20260811 17:18:49.797058 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.796758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:34149 (local address 127.0.0.1:43224)
0811 17:18:49.796914 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:49.796980 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:34149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 17:18:49.797822 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.797600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:34149 (local address 127.0.0.1:43236)
0811 17:18:49.797715 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:49.797767 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:34149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 17:18:49.797955 15265 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.189:34149: Network error: Client connection negotiation failed: client connection to 127.21.54.189:34149: connect: Connection refused (error 111)
W20260811 17:18:49.804325 15274 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 17:18:49.804723 15274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:49.804785 15274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:49.804863 15274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:49.809028 15274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:49.809103 15274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:49.809139 15274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:49.809190 15274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:49.809228 15274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:49.814693 15274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:37271,127.21.54.189:34149,127.21.54.188:33155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.189.keytab
--principal=kudu/127.21.54.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.189:34149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.54.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:49.815832 15274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:49.816955 15274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:49.825780 15282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:49.825781 15281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:49.825780 15284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:49.826562 15274 server_base.cc:1034] running on GCE node
I20260811 17:18:49.827296 15274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:49.828485 15274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:49.829711 15274 hybrid_clock.cc:648] HybridClock initialized: now 1786468729829675 us; error 59 us; skew 500 ppm
Aug 11 17:18:49 dist-test-slave-fd7l krb5kdc[15170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468729, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:49.833088 15274 init.cc:377] Logged in from keytab as kudu/127.21.54.189@KRBTEST.COM (short username kudu)
I20260811 17:18:49.834120 15274 webserver.cc:460] Webserver started at http://127.21.54.189:44343/ using document root <none> and password file <none>
I20260811 17:18:49.834734 15274 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:49.834829 15274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:49.835067 15274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:49.837811 15274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-1/data/instance:
uuid: "ec581d9e2aa740d7951412198479dce2"
format_stamp: "Formatted at 2026-08-11 17:18:49 on dist-test-slave-fd7l"
server_key: "45ee2b8b058d440e15777b9977b732eb"
server_key_iv: "1648600816a6b766f506550594f73410"
server_key_version: "encryptionkey@0"
I20260811 17:18:49.838518 15274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-1/wal/instance:
uuid: "ec581d9e2aa740d7951412198479dce2"
format_stamp: "Formatted at 2026-08-11 17:18:49 on dist-test-slave-fd7l"
server_key: "45ee2b8b058d440e15777b9977b732eb"
server_key_iv: "1648600816a6b766f506550594f73410"
server_key_version: "encryptionkey@0"
I20260811 17:18:49.842613 15274 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:18:49.845232 15293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:49.846616 15274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:49.846737 15274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-1/wal
uuid: "ec581d9e2aa740d7951412198479dce2"
format_stamp: "Formatted at 2026-08-11 17:18:49 on dist-test-slave-fd7l"
server_key: "45ee2b8b058d440e15777b9977b732eb"
server_key_iv: "1648600816a6b766f506550594f73410"
server_key_version: "encryptionkey@0"
I20260811 17:18:49.846834 15274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:49.875212 15274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:49.878737 15274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:49.878930 15274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:49.887154 15274 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.189:34149
I20260811 17:18:49.887465 15352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.189:34149 every 8 connection(s)
I20260811 17:18:49.888447 15274 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-1/data/info.pb
I20260811 17:18:49.889101 15265 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } attempt: 3
I20260811 17:18:49.891831 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 15274
I20260811 17:18:49.891847 15353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:49.891984 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-1/wal/instance
I20260811 17:18:49.892354 21722 external_mini_cluster.cc:1428] Setting key 6fc401a12fa76e243f5d51b35d9d18c1
I20260811 17:18:49.896911 15353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:18:49 dist-test-slave-fd7l krb5kdc[15170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468729, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for kudu/127.21.54.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.54.188@KRBTEST.COM; defaulting to no policy
I20260811 17:18:49.904394 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.889707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:34149 (local address 127.0.0.1:43250)
0811 17:18:49.889887 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:49.889907 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:49.890031 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:49.892867 (+ 2836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:49.892871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:49.893097 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:49.893257 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:49.893266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:49.895372 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:49.895378 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:49.895942 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:49.895954 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:49.896843 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:49.896847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:49.896959 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:49.897565 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:49.897587 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:49.899215 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:49.902385 (+ 3170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:49.902390 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:49.902392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:49.902624 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:49.902985 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:49.902988 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:49.902990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:49.903083 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:49.903907 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:49.903910 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:49.903997 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 17:18:49.904070 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 17:18:49.904245 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 17:18:49.906176 15355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.890523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43250 (local address 127.21.54.189:34149)
0811 17:18:49.890965 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:49.890992 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:49.891043 (+ 51us) server_negotiation.cc:407] Connection header received
0811 17:18:49.891900 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:49.891929 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:49.892446 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:49.892911 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:49.893429 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:49.895201 (+ 1772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:49.896113 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:49.896702 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:49.896881 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:49.899455 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:49.899462 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:49.899480 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:49.899513 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:49.902270 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:49.902747 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:49.902753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:49.902758 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:49.902876 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:49.903181 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:49.903185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:49.903187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:49.903785 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:49.904695 (+ 910us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:49.904914 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 17:18:49.905216 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":134,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.188@KRBTEST.COM" created.
Aug 11 17:18:49 dist-test-slave-fd7l krb5kdc[15170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468729, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.189@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:18:49.908735 15265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:18:49.909672 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.909335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:33155 (local address 127.0.0.1:38170)
0811 17:18:49.909500 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:49.909572 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 17:18:49.910786 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.910443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:33155 (local address 127.0.0.1:38182)
0811 17:18:49.910673 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:49.910724 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
W20260811 17:18:49.910954 15265 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:33155: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
I20260811 17:18:49.914116 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.898966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:37271 (local address 127.0.0.1:38964)
0811 17:18:49.899601 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:49.899675 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:49.899869 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:49.900194 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:49.900201 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:49.900696 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:49.900906 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:49.900915 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:49.902060 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:49.902065 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:49.903204 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:49.903219 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:49.903823 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:49.903827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:49.903955 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:49.904731 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:49.904757 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:49.909035 (+ 4278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:49.912203 (+ 3168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:49.912214 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:49.912220 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:49.912486 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:49.912848 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:49.912850 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:49.912852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:49.912933 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:49.913460 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:49.913466 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:49.913592 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 17:18:49.913762 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 17:18:49.913951 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":135,"threads_started":1}
I20260811 17:18:49.914433 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.899133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38964 (local address 127.21.54.190:37271)
0811 17:18:49.899267 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:49.899273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:49.899759 (+ 486us) server_negotiation.cc:407] Connection header received
0811 17:18:49.899981 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:49.899984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:49.900065 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:49.900159 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:49.901054 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:49.901908 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:49.903336 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:49.903678 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:49.903794 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:49.909290 (+ 5496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:49.909294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:49.909296 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:49.909325 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:49.912060 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:49.912636 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:49.912643 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:49.912648 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:49.912752 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:49.913035 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:49.913041 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:49.913046 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:49.913350 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:49.913457 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:49.913875 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 17:18:49.914046 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:18:49.915545 15353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:18:49 dist-test-slave-fd7l krb5kdc[15170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468729, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.189@KRBTEST.COM for kudu/127.21.54.189@KRBTEST.COM
I20260811 17:18:49.926106 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.916073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:34149 (local address 127.0.0.1:43264)
0811 17:18:49.916201 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:49.916216 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:49.916296 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:49.916658 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:49.916661 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:49.916800 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:49.916897 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:49.916902 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:49.917947 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:49.917949 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:49.918460 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:49.918468 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:49.919056 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:49.919058 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:49.919176 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:49.919587 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:49.919598 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:49.922489 (+ 2891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:49.924544 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:49.924551 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:49.924556 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:49.924767 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:49.925090 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:49.925092 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:49.925094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:49.925177 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:49.925545 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:49.925547 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:49.925663 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 17:18:49.925750 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 17:18:49.925947 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":36}
I20260811 17:18:49.926163 15355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.916243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43264 (local address 127.21.54.189:34149)
0811 17:18:49.916388 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:49.916393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:49.916408 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:18:49.916449 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:49.916453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:49.916500 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:49.916625 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:49.917010 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:49.917840 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:49.918578 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:49.918938 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:49.919068 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:49.922711 (+ 3643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:49.922715 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:49.922717 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:49.922741 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:49.924382 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:49.924876 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:49.924880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:49.924882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:49.924992 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:49.925264 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:49.925267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:49.925269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:49.925464 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:49.925545 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:49.925840 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 17:18:49.926010 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":47}
I20260811 17:18:49.927554 15353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:18:49.928326 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.928078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:33155 (local address 127.0.0.1:38196)
0811 17:18:49.928193 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:49.928270 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 17:18:49.928613 15294 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.188.keytab.
Entry for principal kudu/127.21.54.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.188.keytab.
I20260811 17:18:49.929574 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.929314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:33155 (local address 127.0.0.1:38210)
0811 17:18:49.929431 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:49.929492 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260811 17:18:49.929790 15353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:33155: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
I20260811 17:18:49.930291 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.188:33155
--webserver_interface=127.21.54.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.188.keytab
--principal=kudu/127.21.54.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:37271,127.21.54.189:34149,127.21.54.188:33155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:18:49.942188 15265 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } attempt: 1
I20260811 17:18:49.943346 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.942959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:33155 (local address 127.0.0.1:38220)
0811 17:18:49.943168 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:49.943260 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 17:18:49.944207 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.943971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:33155 (local address 127.0.0.1:38226)
0811 17:18:49.944088 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:49.944142 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 17:18:49.944337 15265 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:33155: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
I20260811 17:18:49.979126 15353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } attempt: 1
I20260811 17:18:49.980104 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.979747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:33155 (local address 127.0.0.1:38242)
0811 17:18:49.979939 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:49.980014 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 17:18:49.981042 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:49.980790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:33155 (local address 127.0.0.1:38252)
0811 17:18:49.980926 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:49.980979 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 17:18:49.981241 15353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:33155: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
I20260811 17:18:50.019570 15265 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } attempt: 2
I20260811 17:18:50.020795 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.020201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:33155 (local address 127.0.0.1:38266)
0811 17:18:50.020423 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:50.020488 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 17:18:50.021654 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.021371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:33155 (local address 127.0.0.1:38268)
0811 17:18:50.021478 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:50.021598 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 17:18:50.021850 15265 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:33155: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
I20260811 17:18:50.049434 15353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } attempt: 2
I20260811 17:18:50.050451 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.050039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:33155 (local address 127.0.0.1:38280)
0811 17:18:50.050301 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:50.050361 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
I20260811 17:18:50.051267 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.051028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:33155 (local address 127.0.0.1:38282)
0811 17:18:50.051140 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:50.051194 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 17:18:50.051512 15353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:33155: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
W20260811 17:18:50.052057 15361 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 17:18:50.052351 15361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:50.052399 15361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:50.052482 15361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:50.056674 15361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:50.056730 15361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:50.056749 15361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:50.056766 15361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:50.056782 15361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:50.062224 15361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:37271,127.21.54.189:34149,127.21.54.188:33155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.188.keytab
--principal=kudu/127.21.54.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.188:33155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.54.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:50.063313 15361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:50.064321 15361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:50.073791 15367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:50.073774 15371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:50.073863 15361 server_base.cc:1034] running on GCE node
W20260811 17:18:50.073796 15369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:50.074632 15361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:50.075771 15361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:50.076994 15361 hybrid_clock.cc:648] HybridClock initialized: now 1786468730076970 us; error 41 us; skew 500 ppm
Aug 11 17:18:50 dist-test-slave-fd7l krb5kdc[15170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468730, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:50.079936 15361 init.cc:377] Logged in from keytab as kudu/127.21.54.188@KRBTEST.COM (short username kudu)
I20260811 17:18:50.081012 15361 webserver.cc:460] Webserver started at http://127.21.54.188:45445/ using document root <none> and password file <none>
I20260811 17:18:50.081642 15361 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:50.081740 15361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:50.082005 15361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:50.084352 15361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-2/data/instance:
uuid: "59ac1f0154ab449b99af103ad32f6a82"
format_stamp: "Formatted at 2026-08-11 17:18:50 on dist-test-slave-fd7l"
server_key: "01b5d57a331c96811cb23fc21626f116"
server_key_iv: "eac5dd12ca07d966f13750ef43f7b7af"
server_key_version: "encryptionkey@0"
I20260811 17:18:50.085057 15361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-2/wal/instance:
uuid: "59ac1f0154ab449b99af103ad32f6a82"
format_stamp: "Formatted at 2026-08-11 17:18:50 on dist-test-slave-fd7l"
server_key: "01b5d57a331c96811cb23fc21626f116"
server_key_iv: "eac5dd12ca07d966f13750ef43f7b7af"
server_key_version: "encryptionkey@0"
I20260811 17:18:50.089211 15361 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:18:50.091840 15379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:50.093291 15361 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:50.093456 15361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-2/wal
uuid: "59ac1f0154ab449b99af103ad32f6a82"
format_stamp: "Formatted at 2026-08-11 17:18:50 on dist-test-slave-fd7l"
server_key: "01b5d57a331c96811cb23fc21626f116"
server_key_iv: "eac5dd12ca07d966f13750ef43f7b7af"
server_key_version: "encryptionkey@0"
I20260811 17:18:50.093590 15361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:50.121043 15265 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } attempt: 3
I20260811 17:18:50.121950 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.121637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:33155 (local address 127.0.0.1:38292)
0811 17:18:50.121786 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:50.121858 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 17:18:50.122786 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.122536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:33155 (local address 127.0.0.1:38302)
0811 17:18:50.122688 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:50.122731 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 17:18:50.122993 15265 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:33155: Network error: Client connection negotiation failed: client connection to 127.21.54.188:33155: connect: Connection refused (error 111)
I20260811 17:18:50.127442 15361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:50.130679 15361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:50.130934 15361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:50.138648 15361 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.188:33155
I20260811 17:18:50.138916 15439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.188:33155 every 8 connection(s)
I20260811 17:18:50.139686 15361 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-2/data/info.pb
I20260811 17:18:50.142616 15440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:50.142728 15353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } attempt: 3
I20260811 17:18:50.147941 15440 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:18:50.148910 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 15361
I20260811 17:18:50.149085 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468428054096-21722-0/minicluster-data/master-2/wal/instance
I20260811 17:18:50.149396 21722 external_mini_cluster.cc:1428] Setting key 2b9fff501936bcab369815e83c0cdb3c
Aug 11 17:18:50 dist-test-slave-fd7l krb5kdc[15170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468729, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.189@KRBTEST.COM for kudu/127.21.54.188@KRBTEST.COM
Aug 11 17:18:50 dist-test-slave-fd7l krb5kdc[15170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468730, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.188@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:18:50.157569 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.143274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:33155 (local address 127.0.0.1:38308)
0811 17:18:50.143432 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:50.143450 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:50.143557 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:50.146127 (+ 2570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:50.146130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:50.146352 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:50.146475 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:50.146482 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.147825 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.147828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:50.148697 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:50.148709 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.149904 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.149911 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:50.150014 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:50.150573 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:50.150590 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:50.152257 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:50.155401 (+ 3144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:50.155405 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:50.155407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:50.155706 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:50.156118 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:50.156122 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:50.156125 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:50.156228 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:50.157077 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:50.157080 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:50.157170 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 17:18:50.157266 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 17:18:50.157412 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
Aug 11 17:18:50 dist-test-slave-fd7l krb5kdc[15170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.188@KRBTEST.COM
I20260811 17:18:50.159225 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.143949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38308 (local address 127.21.54.188:33155)
0811 17:18:50.144349 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.144364 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.144400 (+ 36us) server_negotiation.cc:407] Connection header received
0811 17:18:50.145497 (+ 1097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.145515 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.145836 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.146130 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.146630 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.147682 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.148868 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.149698 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.150404 (+ 706us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:50.152477 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:50.152484 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:50.152497 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:50.152571 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:50.154810 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.155848 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.155853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.155859 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.155988 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.156338 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.156347 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.156349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.156953 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:50.157113 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:50.157532 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 17:18:50.158066 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":126,"threads_started":1}
I20260811 17:18:50.162753 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.150031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38980 (local address 127.21.54.190:37271)
0811 17:18:50.150190 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.150198 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.151427 (+ 1229us) server_negotiation.cc:407] Connection header received
0811 17:18:50.151679 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.151683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.151740 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.151836 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.153142 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.153883 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.154926 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.155173 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.155283 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:50.158638 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:50.158641 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:50.158644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:50.158673 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:50.160649 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.161145 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.161149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.161151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.161247 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.161615 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.161619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.161621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.161837 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:50.161904 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:50.162471 (+ 567us) server_negotiation.cc:299] Negotiation successful
0811 17:18:50.162600 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 17:18:50.162853 15445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.149921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:37271 (local address 127.0.0.1:38980)
0811 17:18:50.150946 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:50.151314 (+ 368us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:50.151547 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:50.151890 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:50.151896 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:50.152433 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:50.152660 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:50.152669 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.153993 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.153998 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:50.154813 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:50.154823 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.155287 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.155291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:50.155408 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:50.155873 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:50.155890 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:50.158421 (+ 2531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:50.160764 (+ 2343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:50.160771 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:50.160776 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:50.161041 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:50.161358 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:50.161365 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:50.161370 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:50.161482 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:50.162018 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:50.162023 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:50.162197 (+ 174us) client_negotiation.cc:769] Sending connection context
0811 17:18:50.162380 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 17:18:50.162597 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"mutex_wait_us":1,"thread_start_us":94,"threads_started":1}
I20260811 17:18:50.164186 15440 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:18:50.164322 15353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2: Bootstrap starting.
I20260811 17:18:50.164667 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.150457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38316 (local address 127.21.54.188:33155)
0811 17:18:50.150796 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.150805 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.150819 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:18:50.150852 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.150855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.150896 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.150989 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.152004 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.152630 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.153843 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.154287 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.154405 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:50.158544 (+ 4139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:50.158548 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:50.158551 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:50.158578 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:50.161981 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.162580 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.162584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.162585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.162685 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.163230 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.163237 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.163239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.163472 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:50.163673 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:50.164264 (+ 591us) server_negotiation.cc:299] Negotiation successful
0811 17:18:50.164383 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":279,"thread_start_us":142,"threads_started":1}
I20260811 17:18:50.167490 15353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:50.168319 15353 log.cc:824] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2: Log is configured to *not* fsync() on all Append() calls
Aug 11 17:18:50 dist-test-slave-fd7l krb5kdc[15170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468730, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.188@KRBTEST.COM for kudu/127.21.54.189@KRBTEST.COM
I20260811 17:18:50.171101 15353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2: No bootstrap required, opened a new log
Aug 11 17:18:50 dist-test-slave-fd7l krb5kdc[15170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:18:50.174296 15445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.164791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:34149 (local address 127.0.0.1:43270)
0811 17:18:50.165294 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:50.165314 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:50.165403 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:50.165668 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:50.165672 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:50.165877 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:50.166016 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:50.166023 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.166981 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.166984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:50.167780 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:50.167792 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.168404 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.168408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:50.168504 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:50.169066 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:50.169082 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:50.170500 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:50.172568 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:50.172572 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:50.172574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:50.172829 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:50.173172 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:50.173175 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:50.173178 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:50.173271 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:50.173757 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:50.173760 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:50.173859 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 17:18:50.173970 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 17:18:50.174126 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425}
I20260811 17:18:50.174808 15355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.164850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43270 (local address 127.21.54.189:34149)
0811 17:18:50.164990 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.164996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.165368 (+ 372us) server_negotiation.cc:407] Connection header received
0811 17:18:50.165498 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.165501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.165552 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.165640 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.166155 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.166847 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.167907 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.168271 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.168383 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:50.170679 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:50.170685 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:50.170689 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:50.170722 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:50.172422 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.172954 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.172958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.172960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.173049 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.173384 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.173389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.173393 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.173645 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:50.173737 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:50.174538 (+ 801us) server_negotiation.cc:299] Negotiation successful
0811 17:18:50.174671 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 17:18:50.175204 15353 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } } peers { permanent_uuid: "ec581d9e2aa740d7951412198479dce2" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } } peers { permanent_uuid: "59ac1f0154ab449b99af103ad32f6a82" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } }
I20260811 17:18:50.175480 15353 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec581d9e2aa740d7951412198479dce2, State: Initialized, Role: FOLLOWER
I20260811 17:18:50.175525 15440 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:18:50.175836 15353 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } } peers { permanent_uuid: "ec581d9e2aa740d7951412198479dce2" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } } peers { permanent_uuid: "59ac1f0154ab449b99af103ad32f6a82" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } }
I20260811 17:18:50.176954 15450 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } } peers { permanent_uuid: "ec581d9e2aa740d7951412198479dce2" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } } peers { permanent_uuid: "59ac1f0154ab449b99af103ad32f6a82" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } } }
I20260811 17:18:50.177155 15450 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:18:50.177426 15353 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:50.178324 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.166005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38994 (local address 127.21.54.190:37271)
0811 17:18:50.166158 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.166163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.166175 (+ 12us) server_negotiation.cc:407] Connection header received
0811 17:18:50.166219 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.166221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.166257 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.166349 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.167022 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.167581 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.168811 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.169176 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.169245 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:50.172970 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:50.172974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:50.172977 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:50.173013 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:50.175459 (+ 2446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.176444 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.176450 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.176452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.176598 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.176978 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.176984 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.176989 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.177297 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:50.177496 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:50.177847 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 17:18:50.177998 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 17:18:50 dist-test-slave-fd7l krb5kdc[15170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468730, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.188@KRBTEST.COM for kudu/127.21.54.188@KRBTEST.COM
W20260811 17:18:50.184105 15467 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:18:50.184197 15467 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:18:50.185447 15445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.176066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:33155 (local address 127.0.0.1:38326)
0811 17:18:50.176197 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:50.176213 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:50.176293 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:50.176658 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:50.176661 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:50.176824 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:50.176914 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:50.176921 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.178007 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.178010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:50.178818 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:50.178829 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.179474 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.179477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:50.179579 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:50.180145 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:50.180159 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:50.181725 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:50.183683 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:50.183686 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:50.183688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:50.183925 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:50.184188 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:50.184191 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:50.184196 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:50.184401 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:50.184949 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:50.184951 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:50.185034 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 17:18:50.185134 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 17:18:50.185273 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 17:18:50.186038 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.176292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38326 (local address 127.21.54.188:33155)
0811 17:18:50.176411 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.176419 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.176432 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:18:50.176467 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.176470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.176548 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.176640 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.177036 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.177777 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.178949 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.179348 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.179457 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:50.181857 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:50.181860 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:50.181863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:50.181883 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:50.183571 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.184019 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.184022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.184024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.184096 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.184499 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.184502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.184522 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.184818 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:50.184914 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:50.185782 (+ 868us) server_negotiation.cc:299] Negotiation successful
0811 17:18:50.185904 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Aug 11 17:18:50 dist-test-slave-fd7l krb5kdc[15170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.189@KRBTEST.COM
I20260811 17:18:50.189904 15440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82: Bootstrap starting.
I20260811 17:18:50.191857 15355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.180080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43272 (local address 127.21.54.189:34149)
0811 17:18:50.180355 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.180360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.180374 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:18:50.180407 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.180410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.180449 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.180644 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.181533 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.182281 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.183428 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.183774 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.183875 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:50.187324 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:50.187332 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:50.187339 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:50.187373 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:50.189286 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.189869 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.189873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.189875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.189992 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.190346 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.190349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.190351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.190593 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:50.190801 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:50.191550 (+ 749us) server_negotiation.cc:299] Negotiation successful
0811 17:18:50.191688 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198}
I20260811 17:18:50.181517 15274 master_runner.cc:186] Error getting master registration for 127.21.54.190:37271: OK, instance_id { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" instance_seqno: 1786468729644665 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 17:18:50.192870 15274 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 17:18:50.194027 15440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:50.195024 15440 log.cc:824] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:50.195739 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:50.198511 15440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82: No bootstrap required, opened a new log
I20260811 17:18:50.197372 15236 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "9a9f5740d9df4eab88769b1d708e050f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38994
I20260811 17:18:50.202874 15440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } } peers { permanent_uuid: "ec581d9e2aa740d7951412198479dce2" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } } peers { permanent_uuid: "59ac1f0154ab449b99af103ad32f6a82" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } }
I20260811 17:18:50.203161 15440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59ac1f0154ab449b99af103ad32f6a82, State: Initialized, Role: FOLLOWER
I20260811 17:18:50.203529 15440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } } peers { permanent_uuid: "ec581d9e2aa740d7951412198479dce2" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } } peers { permanent_uuid: "59ac1f0154ab449b99af103ad32f6a82" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } }
I20260811 17:18:50.204097 15471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } } peers { permanent_uuid: "ec581d9e2aa740d7951412198479dce2" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } } peers { permanent_uuid: "59ac1f0154ab449b99af103ad32f6a82" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } } }
I20260811 17:18:50.204277 15471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:18:50.205026 15440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 17:18:50.211100 15485 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:18:50.211189 15485 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:18:50.209518 15361 master_runner.cc:186] Error getting master registration for 127.21.54.190:37271: OK, instance_id { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" instance_seqno: 1786468729644665 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 17:18:50.213238 15361 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 17:18:50.287197 15265 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } attempt: 4
Aug 11 17:18:50 dist-test-slave-fd7l krb5kdc[15170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468729, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for kudu/127.21.54.188@KRBTEST.COM
I20260811 17:18:50.299134 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.288254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38340 (local address 127.21.54.188:33155)
0811 17:18:50.288395 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.288401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.288423 (+ 22us) server_negotiation.cc:407] Connection header received
0811 17:18:50.288476 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.288482 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.288563 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.288675 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.289242 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.290127 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.291354 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.291709 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.291832 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:50.294657 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:50.294662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:50.294664 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:50.294695 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:50.296652 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.297289 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.297293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.297295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.297389 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.297767 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.297770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.297772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.298033 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:50.298157 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:50.298815 (+ 658us) server_negotiation.cc:299] Negotiation successful
0811 17:18:50.298943 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 17:18:50.299444 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.287868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:33155 (local address 127.0.0.1:38340)
0811 17:18:50.288249 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:50.288267 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:50.288375 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:50.288706 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:50.288709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:50.288953 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:50.289090 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:50.289097 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.290269 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.290273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:50.291209 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:50.291220 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.291848 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.291852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:50.291951 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:50.292618 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:50.292639 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:50.294476 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:50.296795 (+ 2319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:50.296799 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:50.296802 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:50.297162 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:50.297502 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:50.297506 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:50.297508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:50.297657 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:50.298207 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:50.298211 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:50.298323 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 17:18:50.298431 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 17:18:50.298600 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276}
I20260811 17:18:50.303318 15265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f: Bootstrap starting.
I20260811 17:18:50.306507 15265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:50.307559 15265 log.cc:824] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:50.310117 15265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f: No bootstrap required, opened a new log
I20260811 17:18:50.313681 15265 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } } peers { permanent_uuid: "ec581d9e2aa740d7951412198479dce2" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } } peers { permanent_uuid: "59ac1f0154ab449b99af103ad32f6a82" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } }
I20260811 17:18:50.313949 15265 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a9f5740d9df4eab88769b1d708e050f, State: Initialized, Role: FOLLOWER
I20260811 17:18:50.314504 15265 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } } peers { permanent_uuid: "ec581d9e2aa740d7951412198479dce2" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } } peers { permanent_uuid: "59ac1f0154ab449b99af103ad32f6a82" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } }
I20260811 17:18:50.315285 15489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } } peers { permanent_uuid: "ec581d9e2aa740d7951412198479dce2" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } } peers { permanent_uuid: "59ac1f0154ab449b99af103ad32f6a82" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } } }
I20260811 17:18:50.315582 15489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:18:50.316327 15265 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:50.316417 15236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 17:18:50.316577 15236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:50.318071 15236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } } peers { permanent_uuid: "ec581d9e2aa740d7951412198479dce2" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } } peers { permanent_uuid: "59ac1f0154ab449b99af103ad32f6a82" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } }
I20260811 17:18:50.319136 15236 leader_election.cc:288] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [CANDIDATE]: Term 1 election: Requested vote from peers ec581d9e2aa740d7951412198479dce2 (127.21.54.189:34149), 59ac1f0154ab449b99af103ad32f6a82 (127.21.54.188:33155)
I20260811 17:18:50.321192 15325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9a9f5740d9df4eab88769b1d708e050f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ec581d9e2aa740d7951412198479dce2"
I20260811 17:18:50.321467 15325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:50.322921 15325 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a9f5740d9df4eab88769b1d708e050f in term 1.
I20260811 17:18:50.323432 15409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9a9f5740d9df4eab88769b1d708e050f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "59ac1f0154ab449b99af103ad32f6a82"
I20260811 17:18:50.323710 15409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82 [term 0 FOLLOWER]: Advancing to term 1
W20260811 17:18:50.323895 15500 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:18:50.323983 15500 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:18:50.325177 15409 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a9f5740d9df4eab88769b1d708e050f in term 1.
I20260811 17:18:50.325738 15207 leader_election.cc:302] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59ac1f0154ab449b99af103ad32f6a82, 9a9f5740d9df4eab88769b1d708e050f; no voters:
I20260811 17:18:50.330885 15489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:50.331243 15489 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [term 1 LEADER]: Becoming Leader. State: Replica: 9a9f5740d9df4eab88769b1d708e050f, State: Running, Role: LEADER
I20260811 17:18:50.331687 15489 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } } peers { permanent_uuid: "ec581d9e2aa740d7951412198479dce2" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } } peers { permanent_uuid: "59ac1f0154ab449b99af103ad32f6a82" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } }
I20260811 17:18:50.334046 15188 master_runner.cc:263] No leader master found from master 9a9f5740d9df4eab88769b1d708e050f
I20260811 17:18:50.334131 15188 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 17:18:50.334746 15489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a9f5740d9df4eab88769b1d708e050f. Latest consensus state: current_term: 1 leader_uuid: "9a9f5740d9df4eab88769b1d708e050f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } } peers { permanent_uuid: "ec581d9e2aa740d7951412198479dce2" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } } peers { permanent_uuid: "59ac1f0154ab449b99af103ad32f6a82" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } } }
I20260811 17:18:50.334960 15489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:50.344825 15510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:50.345984 15510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:50.347550 15355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.328716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43280 (local address 127.21.54.189:34149)
0811 17:18:50.328855 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.328860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.336908 (+ 8048us) server_negotiation.cc:407] Connection header received
0811 17:18:50.337158 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.337162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.337219 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.337277 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.338474 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.339368 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.340755 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.341130 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.341214 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:50.343382 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:50.343387 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:50.343390 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:50.343422 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:50.345510 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.346073 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.346077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.346079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.346175 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.346501 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.346505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.346507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.346731 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:50.346837 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:50.347242 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 17:18:50.347373 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 17:18:50.351707 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.329712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38342 (local address 127.21.54.188:33155)
0811 17:18:50.329819 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.329823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.341842 (+ 12019us) server_negotiation.cc:407] Connection header received
0811 17:18:50.342018 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.342021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.342074 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.342125 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.343298 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.344064 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.345281 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.345585 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.345684 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:50.347457 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:50.347461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:50.347464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:50.347493 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:50.349634 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.350275 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.350279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.350280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.350373 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.350711 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.350717 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.350720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.350951 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:50.351050 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:50.351416 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 17:18:50.351534 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 17:18:50.359653 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:50.340947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38996 (local address 127.21.54.190:37271)
0811 17:18:50.341071 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.341075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.349497 (+ 8422us) server_negotiation.cc:407] Connection header received
0811 17:18:50.349681 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.349685 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.349737 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.349792 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.350739 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.351658 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.353227 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:50.353611 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:50.353685 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:50.355561 (+ 1876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:50.355566 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:50.355571 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:50.355612 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:50.357550 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.358119 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.358122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.358125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.358221 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:50.358558 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:50.358562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:50.358564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:50.358804 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:50.358915 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:50.359334 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 17:18:50.359474 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 17:18:50.364243 15409 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82 [term 1 FOLLOWER]: Refusing update from remote peer 9a9f5740d9df4eab88769b1d708e050f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:18:50.364319 15325 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2 [term 1 FOLLOWER]: Refusing update from remote peer 9a9f5740d9df4eab88769b1d708e050f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:18:50.365764 15489 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec581d9e2aa740d7951412198479dce2" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:18:50.366807 15489 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [LEADER]: Connected to new peer: Peer: permanent_uuid: "59ac1f0154ab449b99af103ad32f6a82" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:18:50.371157 15471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a9f5740d9df4eab88769b1d708e050f. Latest consensus state: current_term: 1 leader_uuid: "9a9f5740d9df4eab88769b1d708e050f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } } peers { permanent_uuid: "ec581d9e2aa740d7951412198479dce2" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } } peers { permanent_uuid: "59ac1f0154ab449b99af103ad32f6a82" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } } }
I20260811 17:18:50.371292 15471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:18:50.372364 15450 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a9f5740d9df4eab88769b1d708e050f. Latest consensus state: current_term: 1 leader_uuid: "9a9f5740d9df4eab88769b1d708e050f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } } peers { permanent_uuid: "ec581d9e2aa740d7951412198479dce2" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } } peers { permanent_uuid: "59ac1f0154ab449b99af103ad32f6a82" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } } }
I20260811 17:18:50.372520 15450 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:18:50.375800 15489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9a9f5740d9df4eab88769b1d708e050f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } } peers { permanent_uuid: "ec581d9e2aa740d7951412198479dce2" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } } peers { permanent_uuid: "59ac1f0154ab449b99af103ad32f6a82" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } } }
I20260811 17:18:50.375936 15489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:50.376065 15512 mvcc.cc:205] Tried to move back new op lower bound from 7317375919564374016 to 7317375919443206144. Current Snapshot: MvccSnapshot[applied={T|T < 7317375919564374016}]
I20260811 17:18:50.377655 15471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9a9f5740d9df4eab88769b1d708e050f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } } peers { permanent_uuid: "ec581d9e2aa740d7951412198479dce2" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } } peers { permanent_uuid: "59ac1f0154ab449b99af103ad32f6a82" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } } }
I20260811 17:18:50.377776 15471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:18:50.378986 15508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9a9f5740d9df4eab88769b1d708e050f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } } peers { permanent_uuid: "ec581d9e2aa740d7951412198479dce2" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } } peers { permanent_uuid: "59ac1f0154ab449b99af103ad32f6a82" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } } }
I20260811 17:18:50.379130 15508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:50.378443 15513 mvcc.cc:205] Tried to move back new op lower bound from 7317375919564374016 to 7317375919443206144. Current Snapshot: MvccSnapshot[applied={T|T < 7317375919564374016}]
I20260811 17:18:50.380479 15450 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9a9f5740d9df4eab88769b1d708e050f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9f5740d9df4eab88769b1d708e050f" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 37271 } } peers { permanent_uuid: "ec581d9e2aa740d7951412198479dce2" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 34149 } } peers { permanent_uuid: "59ac1f0154ab449b99af103ad32f6a82" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 33155 } } }
I20260811 17:18:50.380645 15450 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:18:50.382311 15510 catalog_manager.cc:1347] Generated new cluster ID: 531b390793084b0482e702fac77c851a
I20260811 17:18:50.382395 15510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:50.409175 15510 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:50.410660 15510 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:50.426928 15510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a9f5740d9df4eab88769b1d708e050f: Generated new TSK 0
I20260811 17:18:50.428014 15510 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 17:18:50.454856 15448 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:50.453478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38356 (local address 127.21.54.188:33155)
0811 17:18:50.453604 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.453609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.453868 (+ 259us) server_negotiation.cc:407] Connection header received
0811 17:18:50.454063 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.454069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.454127 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.454181 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.454707 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38356: BlockingRecv error: recv got EOF from 127.0.0.1:38356 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 17:18:50.454856 15269 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:50.453481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39002 (local address 127.21.54.190:37271)
0811 17:18:50.453613 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.453620 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.453638 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:18:50.453835 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.453842 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.453904 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.453964 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.454720 (+ 756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39002: BlockingRecv error: recv got EOF from 127.0.0.1:39002 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 17:18:50.455214 15355 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:50.453133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43296 (local address 127.21.54.189:34149)
0811 17:18:50.453289 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.453294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.454114 (+ 820us) server_negotiation.cc:407] Connection header received
0811 17:18:50.454346 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.454350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.454410 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.454476 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.455086 (+ 610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43296: BlockingRecv error: recv got EOF from 127.0.0.1:43296 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 17:18:50.459177 15355 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:50.458008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43302 (local address 127.21.54.189:34149)
0811 17:18:50.458119 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.458123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.458181 (+ 58us) server_negotiation.cc:407] Connection header received
0811 17:18:50.458348 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.458351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.458415 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.458533 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.459050 (+ 517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43302: BlockingRecv error: recv got EOF from 127.0.0.1:43302 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260811 17:18:50.463436 15355 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:50.462541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.21.54.189:34149)
0811 17:18:50.462675 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.462680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.462695 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:18:50.462747 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.462751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.462807 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.462908 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.463327 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43312: BlockingRecv error: recv got EOF from 127.0.0.1:43312 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 17:18:50.469152 15355 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:50.468259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43314 (local address 127.21.54.189:34149)
0811 17:18:50.468392 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.468397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.468411 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:18:50.468467 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.468470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.468559 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.468651 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.469043 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43314: BlockingRecv error: recv got EOF from 127.0.0.1:43314 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 17:18:50.476243 15355 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:50.475118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43326 (local address 127.21.54.189:34149)
0811 17:18:50.475313 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.475318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.475419 (+ 101us) server_negotiation.cc:407] Connection header received
0811 17:18:50.475566 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.475569 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.475626 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.475713 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.476120 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43326: BlockingRecv error: recv got EOF from 127.0.0.1:43326 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
W20260811 17:18:50.487602 15355 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:50.486422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43334 (local address 127.21.54.189:34149)
0811 17:18:50.486586 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.486591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.486730 (+ 139us) server_negotiation.cc:407] Connection header received
0811 17:18:50.486845 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.486849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.486910 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.487019 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.487472 (+ 453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43334: BlockingRecv error: recv got EOF from 127.0.0.1:43334 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 17:18:50.507267 15355 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:50.505909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43338 (local address 127.21.54.189:34149)
0811 17:18:50.506078 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.506084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.506222 (+ 138us) server_negotiation.cc:407] Connection header received
0811 17:18:50.506340 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.506344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.506411 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.506593 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.507108 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43338: BlockingRecv error: recv got EOF from 127.0.0.1:43338 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 17:18:50.542928 15355 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:50.541620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43348 (local address 127.21.54.189:34149)
0811 17:18:50.541746 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.541750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.541856 (+ 106us) server_negotiation.cc:407] Connection header received
0811 17:18:50.542038 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.542040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.542092 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.542177 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.542794 (+ 617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43348: BlockingRecv error: recv got EOF from 127.0.0.1:43348 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 17:18:50.610690 15355 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:50.609419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43356 (local address 127.21.54.189:34149)
0811 17:18:50.609581 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.609586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.609832 (+ 246us) server_negotiation.cc:407] Connection header received
0811 17:18:50.609966 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.609969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.610022 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.610116 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.610551 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43356: BlockingRecv error: recv got EOF from 127.0.0.1:43356 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 17:18:50.742324 15355 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:50.740963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43360 (local address 127.21.54.189:34149)
0811 17:18:50.741157 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:50.741162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:50.741294 (+ 132us) server_negotiation.cc:407] Connection header received
0811 17:18:50.741521 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:50.741524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:50.741573 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:50.741680 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:50.742195 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43360: BlockingRecv error: recv got EOF from 127.0.0.1:43360 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 17:18:51.002091 15355 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:51.000603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43372 (local address 127.21.54.189:34149)
0811 17:18:51.000761 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:51.000765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:51.000969 (+ 204us) server_negotiation.cc:407] Connection header received
0811 17:18:51.001099 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:51.001102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:51.001156 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:51.001239 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:51.001945 (+ 706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43372: BlockingRecv error: recv got EOF from 127.0.0.1:43372 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 17:18:51.185751 15467 catalog_manager.cc:1259] Loaded cluster ID: 531b390793084b0482e702fac77c851a
I20260811 17:18:51.185855 15467 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2: loading cluster ID for follower catalog manager: success
I20260811 17:18:51.187839 15467 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2: acquiring CA information for follower catalog manager: success
I20260811 17:18:51.189196 15467 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ec581d9e2aa740d7951412198479dce2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 17:18:51.212872 15485 catalog_manager.cc:1259] Loaded cluster ID: 531b390793084b0482e702fac77c851a
I20260811 17:18:51.212976 15485 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82: loading cluster ID for follower catalog manager: success
I20260811 17:18:51.214831 15485 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82: acquiring CA information for follower catalog manager: success
I20260811 17:18:51.216089 15485 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 59ac1f0154ab449b99af103ad32f6a82: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 17:18:51.520680 15594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:51.516335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43384 (local address 127.21.54.189:34149)
0811 17:18:51.516622 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:51.516628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:51.516717 (+ 89us) server_negotiation.cc:407] Connection header received
0811 17:18:51.516914 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:51.516917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:51.517142 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:51.517276 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:18:51.517564 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:51.518168 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:51.519074 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:51.519336 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:51.519448 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:51.519778 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:18:51.520233 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:18:51.520374 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:51.520411 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 17:18:51.520471 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":77,"threads_started":1}
I20260811 17:18:51.528141 15602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:51.524188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38368 (local address 127.21.54.188:33155)
0811 17:18:51.524417 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:51.524473 (+ 56us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:51.524488 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:18:51.524630 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:51.524633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:51.524753 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:51.524858 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:18:51.525077 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:51.525627 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:51.526577 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:51.526827 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:51.526920 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:51.527249 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:18:51.527684 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:18:51.527777 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:51.527933 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 17:18:51.528009 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":93,"threads_started":1}
I20260811 17:18:51.535964 15594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:51.531527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43398 (local address 127.21.54.189:34149)
0811 17:18:51.531742 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:51.531748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:51.532250 (+ 502us) server_negotiation.cc:407] Connection header received
0811 17:18:51.532405 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:51.532408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:51.532464 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:51.532581 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:18:51.532820 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:51.533485 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:51.534592 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:51.534957 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:51.535053 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:51.535407 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:18:51.535521 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:18:51.535580 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:51.535747 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 17:18:51.535812 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 17:18:51.536473 15602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:51.531781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38372 (local address 127.21.54.188:33155)
0811 17:18:51.531887 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:51.531892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:51.532111 (+ 219us) server_negotiation.cc:407] Connection header received
0811 17:18:51.532167 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:51.532170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:51.532239 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:51.532327 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:18:51.533025 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:51.533812 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:51.534869 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:51.535102 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:51.535212 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:51.535509 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:18:51.535605 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:18:51.535684 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:51.536288 (+ 604us) server_negotiation.cc:299] Negotiation successful
0811 17:18:51.536347 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 17:18:51.537184 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:51.531451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39016 (local address 127.21.54.190:37271)
0811 17:18:51.532064 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:51.532068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:51.532220 (+ 152us) server_negotiation.cc:407] Connection header received
0811 17:18:51.532654 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:51.532657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:51.532778 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:51.532877 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:18:51.533075 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:51.533907 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:51.534916 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:51.535274 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:51.535395 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:51.536123 (+ 728us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:18:51.536561 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:18:51.536639 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:51.536815 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 17:18:51.536891 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":79,"threads_started":1}
I20260811 17:18:51.541539 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 15188
I20260811 17:18:51.552942 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 15274
I20260811 17:18:51.562923 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 15361
2026-08-11T17:18:51Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3474 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:18:51 dist-test-slave-fd7l krb5kdc[15618](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:18:51 dist-test-slave-fd7l krb5kdc[15618](info): set up 2 sockets
Aug 11 17:18:51 dist-test-slave-fd7l krb5kdc[15618](info): commencing operation
krb5kdc: starting...
W20260811 17:18:52.780158 21722 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/test-user.keytab.
Aug 11 17:18:52 dist-test-slave-fd7l krb5kdc[15618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:18:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:18:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.54.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
Entry for principal kudu/127.21.54.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab.
I20260811 17:18:52.924201 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:34033
--webserver_interface=127.21.54.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:34033,127.21.54.189:43061,127.21.54.188:44723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:18:53.047324 15633 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 17:18:53.047791 15633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:53.047864 15633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:53.047950 15633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:53.052294 15633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:53.052405 15633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:53.052444 15633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:53.052485 15633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:53.052564 15633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:53.058382 15633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:34033,127.21.54.189:43061,127.21.54.188:44723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.190.keytab
--principal=kudu/127.21.54.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.190:34033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.54.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:53.059670 15633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:53.060873 15633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:53.070139 15646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:53.070176 15643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:53.070176 15641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:53.070567 15633 server_base.cc:1034] running on GCE node
I20260811 17:18:53.071213 15633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:53.072444 15633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:53.073681 15633 hybrid_clock.cc:648] HybridClock initialized: now 1786468733073668 us; error 28 us; skew 500 ppm
Aug 11 17:18:53 dist-test-slave-fd7l krb5kdc[15618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468733, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:53.076663 15633 init.cc:377] Logged in from keytab as kudu/127.21.54.190@KRBTEST.COM (short username kudu)
I20260811 17:18:53.077652 15633 webserver.cc:460] Webserver started at http://127.21.54.190:33483/ using document root <none> and password file <none>
I20260811 17:18:53.078244 15633 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:53.078337 15633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:53.078573 15633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:53.080745 15633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-0/data/instance:
uuid: "feab9b4abc964873b70961c635dcbfcb"
format_stamp: "Formatted at 2026-08-11 17:18:53 on dist-test-slave-fd7l"
server_key: "f171915a3b424a67950e8e243ad18a55"
server_key_iv: "80ead673a2886ffa06a0dc45a9442c28"
server_key_version: "encryptionkey@0"
I20260811 17:18:53.081327 15633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-0/wal/instance:
uuid: "feab9b4abc964873b70961c635dcbfcb"
format_stamp: "Formatted at 2026-08-11 17:18:53 on dist-test-slave-fd7l"
server_key: "f171915a3b424a67950e8e243ad18a55"
server_key_iv: "80ead673a2886ffa06a0dc45a9442c28"
server_key_version: "encryptionkey@0"
I20260811 17:18:53.085251 15633 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:18:53.087934 15653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:53.089411 15633 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:53.089572 15633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-0/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-0/wal
uuid: "feab9b4abc964873b70961c635dcbfcb"
format_stamp: "Formatted at 2026-08-11 17:18:53 on dist-test-slave-fd7l"
server_key: "f171915a3b424a67950e8e243ad18a55"
server_key_iv: "80ead673a2886ffa06a0dc45a9442c28"
server_key_version: "encryptionkey@0"
I20260811 17:18:53.089704 15633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:53.113500 15633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:53.117056 15633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:53.117281 15633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:53.125346 15633 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.190:34033
I20260811 17:18:53.125777 15712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.190:34033 every 8 connection(s)
I20260811 17:18:53.126442 15633 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-0/data/info.pb
I20260811 17:18:53.129537 15716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:53.133275 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 15633
I20260811 17:18:53.133410 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-0/wal/instance
I20260811 17:18:53.133716 21722 external_mini_cluster.cc:1428] Setting key db5bbb701168604dbf24a40e10fba07f
I20260811 17:18:53.133531 15716 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.54.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.189@KRBTEST.COM" created.
Aug 11 17:18:53 dist-test-slave-fd7l krb5kdc[15618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468733, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:18:53.153893 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.136030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:34033 (local address 127.0.0.1:32812)
0811 17:18:53.136717 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.136783 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:53.138110 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:53.138883 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:53.138894 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:53.139454 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:53.139890 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:53.139915 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.141226 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.141232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:53.142567 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:53.142580 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.143309 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.143316 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:53.143457 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.144191 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:53.144216 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:53.147738 (+ 3522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:53.150265 (+ 2527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:53.150275 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:53.150285 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:53.150546 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:53.151017 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:53.151020 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:53.151022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:53.151178 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:53.152106 (+ 928us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:53.152113 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:53.152272 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 17:18:53.152475 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 17:18:53.153018 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":117,"thread_start_us":128,"threads_started":1}
I20260811 17:18:53.153896 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.136882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32812 (local address 127.21.54.190:34033)
0811 17:18:53.137157 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.137167 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.137203 (+ 36us) server_negotiation.cc:407] Connection header received
0811 17:18:53.138455 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.138460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.138659 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.138806 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.140064 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.141069 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.142704 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.143109 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.143246 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.147911 (+ 4665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:53.147918 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:53.147930 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:53.147952 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:53.150110 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.150712 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.150722 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.150726 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.150883 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.151294 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.151300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.151304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.151952 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:53.152125 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:53.152611 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 17:18:53.152938 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":102,"threads_started":1}
I20260811 17:18:53.157106 15716 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:18:53.157994 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.157676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:43061 (local address 127.0.0.1:43208)
0811 17:18:53.157823 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.157902 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:43061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 17:18:53.158272 15657 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.54.189:43061: connect: Connection refused (error 111)
I20260811 17:18:53.159332 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.159079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:43061 (local address 127.0.0.1:43220)
0811 17:18:53.159206 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.159267 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:43061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 17:18:53.159548 15716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.189:43061: Network error: Client connection negotiation failed: client connection to 127.21.54.189:43061: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.189.keytab.
Entry for principal kudu/127.21.54.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.189.keytab.
I20260811 17:18:53.168334 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.189:43061
--webserver_interface=127.21.54.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.189.keytab
--principal=kudu/127.21.54.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:34033,127.21.54.189:43061,127.21.54.188:44723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:18:53.208840 15716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } attempt: 1
I20260811 17:18:53.209892 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.209533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:43061 (local address 127.0.0.1:43224)
0811 17:18:53.209737 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.209805 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:43061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 17:18:53.210636 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.210420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:43061 (local address 127.0.0.1:43240)
0811 17:18:53.210525 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.210573 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:43061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 17:18:53.210752 15716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.189:43061: Network error: Client connection negotiation failed: client connection to 127.21.54.189:43061: connect: Connection refused (error 111)
I20260811 17:18:53.278923 15716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } attempt: 2
I20260811 17:18:53.279891 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.279554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:43061 (local address 127.0.0.1:43248)
0811 17:18:53.279745 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.279812 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:43061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 17:18:53.280715 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.280428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:43061 (local address 127.0.0.1:43264)
0811 17:18:53.280594 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.280659 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.189:43061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 17:18:53.280835 15716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.189:43061: Network error: Client connection negotiation failed: client connection to 127.21.54.189:43061: connect: Connection refused (error 111)
W20260811 17:18:53.286777 15726 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 17:18:53.287148 15726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:53.287209 15726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:53.287285 15726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:53.291684 15726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:53.291793 15726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:53.291833 15726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:53.291895 15726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:53.291945 15726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:53.298395 15726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:34033,127.21.54.189:43061,127.21.54.188:44723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.189.keytab
--principal=kudu/127.21.54.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.189:43061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.54.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:53.299741 15726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:53.300980 15726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:53.310227 15737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:53.310227 15739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:53.310266 15736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:53.310467 15726 server_base.cc:1034] running on GCE node
I20260811 17:18:53.311197 15726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:53.312394 15726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:53.313601 15726 hybrid_clock.cc:648] HybridClock initialized: now 1786468733313584 us; error 35 us; skew 500 ppm
Aug 11 17:18:53 dist-test-slave-fd7l krb5kdc[15618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468733, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:53.316612 15726 init.cc:377] Logged in from keytab as kudu/127.21.54.189@KRBTEST.COM (short username kudu)
I20260811 17:18:53.317596 15726 webserver.cc:460] Webserver started at http://127.21.54.189:44845/ using document root <none> and password file <none>
I20260811 17:18:53.318185 15726 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:53.318248 15726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:53.318426 15726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:53.320567 15726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-1/data/instance:
uuid: "fa9920ba049149cf8d537dd1566c64d8"
format_stamp: "Formatted at 2026-08-11 17:18:53 on dist-test-slave-fd7l"
server_key: "7209c34078bd4d54f6e6748864862ec2"
server_key_iv: "98ce5ef0a547fcd0c0d4514eceaa8ea1"
server_key_version: "encryptionkey@0"
I20260811 17:18:53.321134 15726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-1/wal/instance:
uuid: "fa9920ba049149cf8d537dd1566c64d8"
format_stamp: "Formatted at 2026-08-11 17:18:53 on dist-test-slave-fd7l"
server_key: "7209c34078bd4d54f6e6748864862ec2"
server_key_iv: "98ce5ef0a547fcd0c0d4514eceaa8ea1"
server_key_version: "encryptionkey@0"
I20260811 17:18:53.325261 15726 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:18:53.327997 15747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:53.329677 15726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:18:53.329818 15726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-1/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-1/wal
uuid: "fa9920ba049149cf8d537dd1566c64d8"
format_stamp: "Formatted at 2026-08-11 17:18:53 on dist-test-slave-fd7l"
server_key: "7209c34078bd4d54f6e6748864862ec2"
server_key_iv: "98ce5ef0a547fcd0c0d4514eceaa8ea1"
server_key_version: "encryptionkey@0"
I20260811 17:18:53.329929 15726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:53.346233 15726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:53.349778 15726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:53.349956 15726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:53.357584 15726 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.189:43061
I20260811 17:18:53.357894 15810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.189:43061 every 8 connection(s)
I20260811 17:18:53.358753 15726 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-1/data/info.pb
I20260811 17:18:53.361862 15811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:53.366066 15811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:18:53.366900 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 15726
I20260811 17:18:53.367038 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-1/wal/instance
I20260811 17:18:53.367362 21722 external_mini_cluster.cc:1428] Setting key 5823e96a5297677edccc5ea24eac04e8
I20260811 17:18:53.372007 15716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } attempt: 3
WARNING: no policy specified for kudu/127.21.54.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.54.188@KRBTEST.COM" created.
Aug 11 17:18:53 dist-test-slave-fd7l krb5kdc[15618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468733, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.189@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
Aug 11 17:18:53 dist-test-slave-fd7l krb5kdc[15618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468733, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for kudu/127.21.54.189@KRBTEST.COM
I20260811 17:18:53.385767 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.372485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:43061 (local address 127.0.0.1:46816)
0811 17:18:53.372707 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.372721 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:53.372794 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:53.373516 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:53.373522 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:53.373752 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:53.373905 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:53.373916 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.375023 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.375026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:53.375700 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:53.375732 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.376311 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.376315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:53.376434 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.376975 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:53.376989 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:53.381503 (+ 4514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:53.383625 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:53.383631 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:53.383634 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:53.383917 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:53.384282 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:53.384287 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:53.384289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:53.384370 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:53.385137 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:53.385141 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:53.385243 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 17:18:53.385354 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 17:18:53.385562 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":4}
I20260811 17:18:53.385767 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.368465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38820 (local address 127.21.54.190:34033)
0811 17:18:53.368693 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.368700 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.369599 (+ 899us) server_negotiation.cc:407] Connection header received
0811 17:18:53.370680 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.370687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.370767 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.370858 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.372110 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.372887 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.374322 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.374773 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.374901 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.381328 (+ 6427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:53.381331 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:53.381334 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:53.381360 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:53.383494 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.383998 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.384006 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.384011 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.384110 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.384543 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.384546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.384548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.384776 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:53.384881 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:53.385398 (+ 517us) server_negotiation.cc:299] Negotiation successful
0811 17:18:53.385630 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":113,"server-negotiator.queue_time_us":83}
I20260811 17:18:53.386829 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.372617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46816 (local address 127.21.54.189:43061)
0811 17:18:53.373046 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.373060 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.373082 (+ 22us) server_negotiation.cc:407] Connection header received
0811 17:18:53.373123 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.373128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.373323 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.373492 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.374066 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.374889 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.375835 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.376187 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.376304 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.381735 (+ 5431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:53.381746 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:53.381766 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:53.381798 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:53.383493 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.384046 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.384055 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.384059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.384199 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.384465 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.384468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.384470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.385024 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:53.385167 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:53.386416 (+ 1249us) server_negotiation.cc:299] Negotiation successful
0811 17:18:53.386537 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":158,"threads_started":1}
I20260811 17:18:53.386838 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.368918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:34033 (local address 127.0.0.1:38820)
0811 17:18:53.369435 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.369489 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:53.370453 (+ 964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:53.371024 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:53.371031 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:53.371399 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:53.371948 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:53.371963 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.373029 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.373033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:53.374192 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:53.374203 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.374952 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.374956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:53.375190 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.376495 (+ 1305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:53.376520 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:53.381152 (+ 4632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:53.383630 (+ 2478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:53.383640 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:53.383652 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:53.383887 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:53.384239 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:53.384244 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:53.384249 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:53.384421 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:53.384905 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:53.384914 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:53.385128 (+ 214us) client_negotiation.cc:769] Sending connection context
0811 17:18:53.385342 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 17:18:53.385712 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":157,"threads_started":1}
I20260811 17:18:53.388445 15811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:18:53.389048 15716 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:18:53.389863 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.389539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44723 (local address 127.0.0.1:36516)
0811 17:18:53.389729 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.389786 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 17:18:53.390570 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.390314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44723 (local address 127.0.0.1:36532)
0811 17:18:53.390449 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.390501 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 17:18:53.390822 15716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:44723: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
Aug 11 17:18:53 dist-test-slave-fd7l krb5kdc[15618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468733, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.189@KRBTEST.COM for kudu/127.21.54.189@KRBTEST.COM
I20260811 17:18:53.399235 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.389470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46820 (local address 127.21.54.189:43061)
0811 17:18:53.389603 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.389607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.389855 (+ 248us) server_negotiation.cc:407] Connection header received
0811 17:18:53.389882 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.389885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.389926 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.389984 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.390606 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.391257 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.392435 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.392751 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.392830 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.395435 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:53.395439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:53.395441 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:53.395464 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:53.397227 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.397743 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.397746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.397747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.397870 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.398228 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.398231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.398232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.398428 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:53.398554 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:53.398902 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 17:18:53.399040 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 17:18:53.399263 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.389189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:43061 (local address 127.0.0.1:46820)
0811 17:18:53.389658 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.389674 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:53.389774 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:53.390103 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:53.390107 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:53.390306 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:53.390468 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:53.390479 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.391374 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.391381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:53.392309 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:53.392321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.392861 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.392864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:53.392983 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.393491 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:53.393504 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:53.395273 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:53.397362 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:53.397365 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:53.397366 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:53.397621 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:53.398036 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:53.398039 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:53.398041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:53.398133 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:53.398601 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:53.398605 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:53.398711 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 17:18:53.398805 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 17:18:53.398931 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392}
I20260811 17:18:53.400180 15811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:18:53.401157 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.400874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44723 (local address 127.0.0.1:36540)
0811 17:18:53.400999 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.401089 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 17:18:53.401886 15751 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
I20260811 17:18:53.402989 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.402770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44723 (local address 127.0.0.1:36548)
0811 17:18:53.402885 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.402932 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 17:18:53.403232 15811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:44723: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.54.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.188.keytab.
Entry for principal kudu/127.21.54.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.188.keytab.
I20260811 17:18:53.404801 21722 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2dz75L/build/debug/bin/kudu
/tmp/dist-test-task2dz75L/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.188:44723
--webserver_interface=127.21.54.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.188.keytab
--principal=kudu/127.21.54.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.54.148:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.54.190:34033,127.21.54.189:43061,127.21.54.188:44723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:18:53.422014 15716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } attempt: 1
I20260811 17:18:53.422948 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.422609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44723 (local address 127.0.0.1:36560)
0811 17:18:53.422798 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.422866 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 17:18:53.423700 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.423487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44723 (local address 127.0.0.1:36570)
0811 17:18:53.423594 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.423644 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 17:18:53.423909 15716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:44723: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
I20260811 17:18:53.452569 15811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } attempt: 1
I20260811 17:18:53.453498 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.453190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44723 (local address 127.0.0.1:36576)
0811 17:18:53.453352 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.453417 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 17:18:53.454296 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.454097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44723 (local address 127.0.0.1:36582)
0811 17:18:53.454202 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.454246 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 17:18:53.454445 15811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:44723: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
I20260811 17:18:53.499086 15716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } attempt: 2
I20260811 17:18:53.500006 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.499696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44723 (local address 127.0.0.1:36590)
0811 17:18:53.499858 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.499924 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 17:18:53.500865 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.500634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44723 (local address 127.0.0.1:36598)
0811 17:18:53.500741 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.500809 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 17:18:53.500998 15716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:44723: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
I20260811 17:18:53.522605 15811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } attempt: 2
I20260811 17:18:53.523614 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.523226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44723 (local address 127.0.0.1:36600)
0811 17:18:53.523422 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.523511 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 17:18:53.524609 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.524226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44723 (local address 127.0.0.1:36610)
0811 17:18:53.524439 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.524496 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 17:18:53.524827 15811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:44723: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
W20260811 17:18:53.531172 15820 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 17:18:53.531548 15820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:18:53.531594 15820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:18:53.531665 15820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:18:53.535847 15820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:18:53.535916 15820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:18:53.535938 15820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:18:53.535955 15820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:18:53.536010 15820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:18:53.541788 15820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.54.148:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.54.190:34033,127.21.54.189:43061,127.21.54.188:44723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/krb5kdc/kudu_127.21.54.188.keytab
--principal=kudu/127.21.54.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.54.188:44723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.54.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:18:53.542959 15820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:18:53.544034 15820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:18:53.552739 15827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:53.552773 15826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:18:53.552758 15831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:18:53.553109 15820 server_base.cc:1034] running on GCE node
I20260811 17:18:53.553702 15820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:18:53.554816 15820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:18:53.555971 15820 hybrid_clock.cc:648] HybridClock initialized: now 1786468733555943 us; error 44 us; skew 500 ppm
Aug 11 17:18:53 dist-test-slave-fd7l krb5kdc[15618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468733, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:18:53.558984 15820 init.cc:377] Logged in from keytab as kudu/127.21.54.188@KRBTEST.COM (short username kudu)
I20260811 17:18:53.560030 15820 webserver.cc:460] Webserver started at http://127.21.54.188:38267/ using document root <none> and password file <none>
I20260811 17:18:53.560722 15820 fs_manager.cc:359] Metadata directory not provided
I20260811 17:18:53.560797 15820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:18:53.561038 15820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:18:53.563267 15820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-2/data/instance:
uuid: "5ca7102b594f4fe790e04ab35a3557b9"
format_stamp: "Formatted at 2026-08-11 17:18:53 on dist-test-slave-fd7l"
server_key: "df82e87829b64a638d906dc5010b1b0f"
server_key_iv: "c68994cc586bd00dfebe2e3a07ad26ea"
server_key_version: "encryptionkey@0"
I20260811 17:18:53.563887 15820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-2/wal/instance:
uuid: "5ca7102b594f4fe790e04ab35a3557b9"
format_stamp: "Formatted at 2026-08-11 17:18:53 on dist-test-slave-fd7l"
server_key: "df82e87829b64a638d906dc5010b1b0f"
server_key_iv: "c68994cc586bd00dfebe2e3a07ad26ea"
server_key_version: "encryptionkey@0"
I20260811 17:18:53.568024 15820 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 17:18:53.570602 15838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:18:53.571943 15820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 17:18:53.572098 15820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-2/data,/tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-2/wal
uuid: "5ca7102b594f4fe790e04ab35a3557b9"
format_stamp: "Formatted at 2026-08-11 17:18:53 on dist-test-slave-fd7l"
server_key: "df82e87829b64a638d906dc5010b1b0f"
server_key_iv: "c68994cc586bd00dfebe2e3a07ad26ea"
server_key_version: "encryptionkey@0"
I20260811 17:18:53.572219 15820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:18:53.600178 15716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } attempt: 3
I20260811 17:18:53.601094 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.600773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44723 (local address 127.0.0.1:36618)
0811 17:18:53.600934 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.601006 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 17:18:53.601851 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.601643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44723 (local address 127.0.0.1:36630)
0811 17:18:53.601749 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.601799 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 17:18:53.602005 15716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:44723: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
I20260811 17:18:53.616039 15811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } attempt: 3
I20260811 17:18:53.617064 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.616754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44723 (local address 127.0.0.1:36632)
0811 17:18:53.616905 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.616979 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 17:18:53.618057 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.617734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44723 (local address 127.0.0.1:36644)
0811 17:18:53.617941 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.617991 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260811 17:18:53.618283 15811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.54.188:44723: Network error: Client connection negotiation failed: client connection to 127.21.54.188:44723: connect: Connection refused (error 111)
I20260811 17:18:53.620091 15820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:18:53.623515 15820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:18:53.623729 15820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:18:53.630834 15820 rpc_server.cc:307] RPC server started. Bound to: 127.21.54.188:44723
I20260811 17:18:53.631142 15903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.54.188:44723 every 8 connection(s)
I20260811 17:18:53.631865 15820 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-2/data/info.pb
I20260811 17:18:53.633045 21722 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2dz75L/build/debug/bin/kudu as pid 15820
I20260811 17:18:53.633205 21722 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2dz75L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468428054096-21722-0/minicluster-data/master-2/wal/instance
I20260811 17:18:53.633543 21722 external_mini_cluster.cc:1428] Setting key f5a8c252039c6049a7ba47ef2b213125
I20260811 17:18:53.635308 15905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:18:53.640196 15905 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:18:53 dist-test-slave-fd7l krb5kdc[15618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.188@KRBTEST.COM
Aug 11 17:18:53 dist-test-slave-fd7l krb5kdc[15618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468733, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.188@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
I20260811 17:18:53.650225 15908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.634968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36654 (local address 127.21.54.188:44723)
0811 17:18:53.635429 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.635451 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.635496 (+ 45us) server_negotiation.cc:407] Connection header received
0811 17:18:53.636424 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.636449 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.636915 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.637321 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.638048 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.639613 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.640826 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.641528 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.641701 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.644692 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:53.644697 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:53.644706 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:53.644733 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:53.646701 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.647273 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.647280 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.647286 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.647436 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.647724 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.647727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.647729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.648255 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:53.648405 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:53.648773 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 17:18:53.649148 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":135,"threads_started":1}
I20260811 17:18:53.653072 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.642339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38834 (local address 127.21.54.190:34033)
0811 17:18:53.642510 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.642517 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.642914 (+ 397us) server_negotiation.cc:407] Connection header received
0811 17:18:53.643056 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.643061 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.643130 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.643232 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.644187 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.644862 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.646188 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.646402 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.646488 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.649332 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:53.649336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:53.649339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:53.649368 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:53.650984 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.651606 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.651609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.651610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.651695 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.651997 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.652000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.652002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.652239 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:53.652341 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:53.652790 (+ 449us) server_negotiation.cc:299] Negotiation successful
0811 17:18:53.652905 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 17:18:53.653084 15909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.642297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.190:34033 (local address 127.0.0.1:38834)
0811 17:18:53.642769 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.642818 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:53.642948 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:53.643331 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:53.643337 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:53.643798 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:53.643950 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:53.643957 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.644988 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.644995 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:53.646058 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:53.646073 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.646519 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.646523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:53.646670 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.647292 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:53.647342 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:53.649194 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:53.651105 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:53.651115 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:53.651122 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:53.651429 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:53.651795 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:53.651797 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:53.651799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:53.651893 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:53.652334 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:53.652340 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:53.652454 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 17:18:53.652716 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 17:18:53.652903 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"spinlock_wait_cycles":2432,"thread_start_us":148,"threads_started":1}
I20260811 17:18:53.654094 15905 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:18:53 dist-test-slave-fd7l krb5kdc[15618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.190@KRBTEST.COM
Aug 11 17:18:53 dist-test-slave-fd7l krb5kdc[15618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468733, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.188@KRBTEST.COM for kudu/127.21.54.189@KRBTEST.COM
I20260811 17:18:53.663393 15909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.654597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.189:43061 (local address 127.0.0.1:46824)
0811 17:18:53.654743 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.654756 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:53.654820 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:53.655044 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:53.655047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:53.655216 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:53.655340 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:53.655346 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.656295 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.656298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:53.657104 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:53.657116 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.657761 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.657763 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:53.657836 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.658266 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:53.658285 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:53.659776 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:53.661665 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:53.661669 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:53.661671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:53.661922 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:53.662253 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:53.662256 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:53.662258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:53.662357 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:53.662794 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:53.662797 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:53.662910 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 17:18:53.663025 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 17:18:53.663195 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 17:18:53.663430 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.654708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46824 (local address 127.21.54.189:43061)
0811 17:18:53.654830 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.654835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.654854 (+ 19us) server_negotiation.cc:407] Connection header received
0811 17:18:53.654891 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.654893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.654932 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.654997 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.655454 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.656191 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.657257 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.657636 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.657769 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.659947 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:53.659952 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:53.659957 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:53.660006 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:53.661556 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.662016 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.662021 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.662022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.662151 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.662459 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.662463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.662464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.662671 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:53.662782 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:53.663158 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 17:18:53.663275 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 17:18:53.663455 15910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.652748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38844 (local address 127.21.54.190:34033)
0811 17:18:53.653036 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.653048 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.653494 (+ 446us) server_negotiation.cc:407] Connection header received
0811 17:18:53.653535 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.653538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.653584 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.653670 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.654358 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.655166 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.656234 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.656625 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.656733 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.659474 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:53.659478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:53.659480 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:53.659505 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:53.661406 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.661909 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.661912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.661913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.662015 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.662350 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.662352 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.662353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.662710 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:53.662974 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:53.663101 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 17:18:53.663261 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":91,"threads_started":1}
I20260811 17:18:53.664197 15905 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:18:53 dist-test-slave-fd7l krb5kdc[15618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468733, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.188@KRBTEST.COM for kudu/127.21.54.188@KRBTEST.COM
Aug 11 17:18:53 dist-test-slave-fd7l krb5kdc[15618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.54.189@KRBTEST.COM
I20260811 17:18:53.673417 15909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.664812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44723 (local address 127.0.0.1:36666)
0811 17:18:53.664930 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.664944 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:53.665018 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:53.665444 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:53.665445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:53.665581 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:53.665657 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:53.665662 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.666380 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.666386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:53.667001 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:53.667011 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.668083 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.668086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:53.668155 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.668717 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:53.668728 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:53.670009 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:53.671856 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:53.671862 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:53.671865 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:53.672049 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:53.672332 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:53.672337 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:53.672339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:53.672413 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:53.672886 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:53.672888 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:53.672995 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 17:18:53.673116 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 17:18:53.673287 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 17:18:53.673614 15908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.664848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36666 (local address 127.21.54.188:44723)
0811 17:18:53.665162 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.665166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.665178 (+ 12us) server_negotiation.cc:407] Connection header received
0811 17:18:53.665206 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.665208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.665271 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.665349 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.665766 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.666277 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.667112 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.667962 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.668084 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.670156 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:53.670159 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:53.670161 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:53.670183 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:53.671748 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.672159 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.672162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.672164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.672239 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.672564 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.672571 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.672573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.672758 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:53.672891 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:53.673175 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 17:18:53.673360 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":41}
I20260811 17:18:53.674352 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.665415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46834 (local address 127.21.54.189:43061)
0811 17:18:53.665533 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.665537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.665550 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:18:53.665742 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.665745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.665789 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.665873 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.666501 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.666983 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.667752 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.667976 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.668102 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.670757 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:53.670761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:53.670767 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:53.670791 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:53.672375 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.672823 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.672827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.672832 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.672930 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.673257 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.673261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.673262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.673488 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:53.673738 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:53.674080 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 17:18:53.674203 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 17:18:53.677253 15905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9: Bootstrap starting.
I20260811 17:18:53.678277 21722 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:18:53.679898 15905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:53.679461 15682 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "feab9b4abc964873b70961c635dcbfcb"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38844
I20260811 17:18:53.680727 15905 log.cc:824] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:53.683051 15905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9: No bootstrap required, opened a new log
I20260811 17:18:53.686167 15905 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } } peers { permanent_uuid: "fa9920ba049149cf8d537dd1566c64d8" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } } peers { permanent_uuid: "5ca7102b594f4fe790e04ab35a3557b9" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } }
I20260811 17:18:53.686429 15905 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ca7102b594f4fe790e04ab35a3557b9, State: Initialized, Role: FOLLOWER
I20260811 17:18:53.686775 15905 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } } peers { permanent_uuid: "fa9920ba049149cf8d537dd1566c64d8" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } } peers { permanent_uuid: "5ca7102b594f4fe790e04ab35a3557b9" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } }
I20260811 17:18:53.687278 15912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } } peers { permanent_uuid: "fa9920ba049149cf8d537dd1566c64d8" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } } peers { permanent_uuid: "5ca7102b594f4fe790e04ab35a3557b9" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } } }
I20260811 17:18:53.687423 15912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:18:53.688256 15905 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 17:18:53.693567 15924 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:18:53.693665 15924 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:18:53.692139 15820 master_runner.cc:186] Error getting master registration for 127.21.54.190:34033: OK, instance_id { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" instance_seqno: 1786468733123151 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 17:18:53.695830 15820 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 17:18:53.761453 15811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } attempt: 4
I20260811 17:18:53.766171 15716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } attempt: 4
Aug 11 17:18:53 dist-test-slave-fd7l krb5kdc[15618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468733, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.189@KRBTEST.COM for kudu/127.21.54.188@KRBTEST.COM
Aug 11 17:18:53 dist-test-slave-fd7l krb5kdc[15618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468733, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.54.190@KRBTEST.COM for kudu/127.21.54.188@KRBTEST.COM
I20260811 17:18:53.777060 15908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.762295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36682 (local address 127.21.54.188:44723)
0811 17:18:53.762522 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.762528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.762545 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:18:53.762687 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.762690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.762745 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.762851 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.763315 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.763829 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.764692 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.764935 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.765062 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.772402 (+ 7340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:53.772407 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:53.772410 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:53.772436 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:53.774464 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.775149 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.775152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.775154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.775296 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.775966 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.775972 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.775975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.776259 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:53.776406 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:53.776762 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 17:18:53.776881 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260811 17:18:53.777060 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.762260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44723 (local address 127.0.0.1:36682)
0811 17:18:53.762413 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.762432 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:53.762558 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:53.762873 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:53.762876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:53.763091 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:53.763187 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:53.763193 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.763959 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.763963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:53.764502 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:53.764573 (+ 71us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.765064 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.765068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:53.765179 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.770552 (+ 5373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:53.770568 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:53.772216 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:53.774636 (+ 2420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:53.774642 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:53.774644 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:53.775003 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:53.775420 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:53.775424 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:53.775426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:53.775548 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:53.776404 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:53.776425 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:53.776563 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 17:18:53.776695 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 17:18:53.776872 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 17:18:53.777354 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.766991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.54.188:44723 (local address 127.0.0.1:36688)
0811 17:18:53.767229 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 17:18:53.767243 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:18:53.767341 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:18:53.767901 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:18:53.767905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:18:53.768151 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:18:53.768272 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:53.768280 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.769464 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.769468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:53.770378 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:18:53.770393 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.771077 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.771081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:18:53.771185 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.771733 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:18:53.771749 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:18:53.773322 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:18:53.775551 (+ 2229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:53.775555 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:53.775557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:53.775844 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:53.776198 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:18:53.776201 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:18:53.776203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:18:53.776309 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:18:53.776818 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:18:53.776822 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:18:53.776930 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 17:18:53.777016 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 17:18:53.777181 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149}
I20260811 17:18:53.778216 15926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.767075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36688 (local address 127.21.54.188:44723)
0811 17:18:53.767584 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.767606 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.767623 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:18:53.767673 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.767676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.767736 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.767843 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.768403 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.769325 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.770543 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.770927 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.771436 (+ 509us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.773498 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:53.773502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:53.773504 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:53.773532 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:53.775383 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.775951 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.775955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.775957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.776064 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.776428 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.776431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.776433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.776696 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:53.777663 (+ 967us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:53.777744 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 17:18:53.777882 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":140,"threads_started":1}
I20260811 17:18:53.781226 15811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8: Bootstrap starting.
I20260811 17:18:53.783787 15811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:53.785298 15811 log.cc:824] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:53.787672 15716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb: Bootstrap starting.
I20260811 17:18:53.790589 15811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8: No bootstrap required, opened a new log
I20260811 17:18:53.790728 15716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb: Neither blocks nor log segments found. Creating new log.
I20260811 17:18:53.791492 15716 log.cc:824] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb: Log is configured to *not* fsync() on all Append() calls
I20260811 17:18:53.794049 15716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb: No bootstrap required, opened a new log
I20260811 17:18:53.795110 15811 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } } peers { permanent_uuid: "fa9920ba049149cf8d537dd1566c64d8" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } } peers { permanent_uuid: "5ca7102b594f4fe790e04ab35a3557b9" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } }
I20260811 17:18:53.795380 15811 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9920ba049149cf8d537dd1566c64d8, State: Initialized, Role: FOLLOWER
I20260811 17:18:53.795739 15811 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } } peers { permanent_uuid: "fa9920ba049149cf8d537dd1566c64d8" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } } peers { permanent_uuid: "5ca7102b594f4fe790e04ab35a3557b9" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } }
I20260811 17:18:53.797341 15811 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:53.797353 15716 raft_consensus.cc:348] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } } peers { permanent_uuid: "fa9920ba049149cf8d537dd1566c64d8" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } } peers { permanent_uuid: "5ca7102b594f4fe790e04ab35a3557b9" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } }
I20260811 17:18:53.797683 15716 raft_consensus.cc:729] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feab9b4abc964873b70961c635dcbfcb, State: Initialized, Role: FOLLOWER
I20260811 17:18:53.798043 15716 consensus_queue.cc:260] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } } peers { permanent_uuid: "fa9920ba049149cf8d537dd1566c64d8" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } } peers { permanent_uuid: "5ca7102b594f4fe790e04ab35a3557b9" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } }
I20260811 17:18:53.798708 15932 sys_catalog.cc:455] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } } peers { permanent_uuid: "fa9920ba049149cf8d537dd1566c64d8" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } } peers { permanent_uuid: "5ca7102b594f4fe790e04ab35a3557b9" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } } }
I20260811 17:18:53.798985 15932 sys_catalog.cc:458] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:18:53.799669 15682 raft_consensus.cc:482] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 17:18:53.799793 15716 sys_catalog.cc:564] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:18:53.799794 15682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:53.802268 15929 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } } peers { permanent_uuid: "fa9920ba049149cf8d537dd1566c64d8" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } } peers { permanent_uuid: "5ca7102b594f4fe790e04ab35a3557b9" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } } }
I20260811 17:18:53.802410 15929 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:18:53.802744 15682 raft_consensus.cc:504] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } } peers { permanent_uuid: "fa9920ba049149cf8d537dd1566c64d8" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } } peers { permanent_uuid: "5ca7102b594f4fe790e04ab35a3557b9" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } }
W20260811 17:18:53.802909 15942 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:18:53.802981 15942 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:18:53.803841 15682 leader_election.cc:288] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [CANDIDATE]: Term 1 election: Requested vote from peers fa9920ba049149cf8d537dd1566c64d8 (127.21.54.189:43061), 5ca7102b594f4fe790e04ab35a3557b9 (127.21.54.188:44723)
I20260811 17:18:53.804857 15777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "feab9b4abc964873b70961c635dcbfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fa9920ba049149cf8d537dd1566c64d8"
I20260811 17:18:53.805102 15777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:18:53.806552 15777 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate feab9b4abc964873b70961c635dcbfcb in term 1.
I20260811 17:18:53.807045 15870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "feab9b4abc964873b70961c635dcbfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5ca7102b594f4fe790e04ab35a3557b9"
I20260811 17:18:53.807315 15870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9 [term 0 FOLLOWER]: Advancing to term 1
W20260811 17:18:53.807495 15956 catalog_manager.cc:1558] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:18:53.807561 15956 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:18:53.808439 15870 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate feab9b4abc964873b70961c635dcbfcb in term 1.
I20260811 17:18:53.808976 15657 leader_election.cc:302] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ca7102b594f4fe790e04ab35a3557b9, feab9b4abc964873b70961c635dcbfcb; no voters:
I20260811 17:18:53.812664 15932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:18:53.812888 15932 raft_consensus.cc:686] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [term 1 LEADER]: Becoming Leader. State: Replica: feab9b4abc964873b70961c635dcbfcb, State: Running, Role: LEADER
I20260811 17:18:53.813232 15932 consensus_queue.cc:237] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } } peers { permanent_uuid: "fa9920ba049149cf8d537dd1566c64d8" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } } peers { permanent_uuid: "5ca7102b594f4fe790e04ab35a3557b9" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } }
I20260811 17:18:53.815151 15932 sys_catalog.cc:455] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader feab9b4abc964873b70961c635dcbfcb. Latest consensus state: current_term: 1 leader_uuid: "feab9b4abc964873b70961c635dcbfcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } } peers { permanent_uuid: "fa9920ba049149cf8d537dd1566c64d8" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } } peers { permanent_uuid: "5ca7102b594f4fe790e04ab35a3557b9" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } } }
I20260811 17:18:53.815260 15932 sys_catalog.cc:458] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:53.816663 15633 master_runner.cc:263] No leader master found from master feab9b4abc964873b70961c635dcbfcb
I20260811 17:18:53.816735 15633 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 17:18:53.817001 15970 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:18:53.810161 15726 master_runner.cc:186] Error getting master registration for 127.21.54.190:34033: OK, instance_id { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" instance_seqno: 1786468733123151 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 17:18:53.817337 15726 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 17:18:53.817948 15970 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:18:53.823379 15870 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9 [term 1 FOLLOWER]: Refusing update from remote peer feab9b4abc964873b70961c635dcbfcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:18:53.823380 15777 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8 [term 1 FOLLOWER]: Refusing update from remote peer feab9b4abc964873b70961c635dcbfcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:18:53.824272 15964 consensus_queue.cc:1035] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ca7102b594f4fe790e04ab35a3557b9" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:18:53.824465 15932 consensus_queue.cc:1035] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9920ba049149cf8d537dd1566c64d8" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:18:53.829188 15929 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader feab9b4abc964873b70961c635dcbfcb. Latest consensus state: current_term: 1 leader_uuid: "feab9b4abc964873b70961c635dcbfcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } } peers { permanent_uuid: "fa9920ba049149cf8d537dd1566c64d8" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } } peers { permanent_uuid: "5ca7102b594f4fe790e04ab35a3557b9" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } } }
I20260811 17:18:53.829340 15929 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:18:53.833835 15912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader feab9b4abc964873b70961c635dcbfcb. Latest consensus state: current_term: 1 leader_uuid: "feab9b4abc964873b70961c635dcbfcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } } peers { permanent_uuid: "fa9920ba049149cf8d537dd1566c64d8" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } } peers { permanent_uuid: "5ca7102b594f4fe790e04ab35a3557b9" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } } }
I20260811 17:18:53.833983 15912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:18:53.838590 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.812340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46838 (local address 127.21.54.189:43061)
0811 17:18:53.812460 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.812464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.820846 (+ 8382us) server_negotiation.cc:407] Connection header received
0811 17:18:53.821071 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.821074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.821139 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.821196 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.824742 (+ 3546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.825563 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.828817 (+ 3254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.829144 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.829226 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.834870 (+ 5644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:53.834877 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:53.834880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:53.834914 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:53.836696 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.837216 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.837220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.837222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.837309 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.837594 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.837598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.837599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.837820 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:53.837934 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:53.838299 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 17:18:53.838429 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 17:18:53.841363 15926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.813088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.21.54.188:44723)
0811 17:18:53.813218 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.813222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.822508 (+ 9286us) server_negotiation.cc:407] Connection header received
0811 17:18:53.822703 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.822707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.822765 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.822818 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.825716 (+ 2898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.826511 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.830015 (+ 3504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.830375 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.830456 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.835301 (+ 4845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:53.835305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:53.835308 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:53.835337 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:53.838034 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.838571 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.838576 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.838578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.838672 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.839078 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.839081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.839083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.839321 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:53.839451 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:53.841037 (+ 1586us) server_negotiation.cc:299] Negotiation successful
0811 17:18:53.841192 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 17:18:53.846307 15971 mvcc.cc:205] Tried to move back new op lower bound from 7317375933733171200 to 7317375933702688768. Current Snapshot: MvccSnapshot[applied={T|T < 7317375933733171200}]
I20260811 17:18:53.846918 15932 sys_catalog.cc:455] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "feab9b4abc964873b70961c635dcbfcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } } peers { permanent_uuid: "fa9920ba049149cf8d537dd1566c64d8" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } } peers { permanent_uuid: "5ca7102b594f4fe790e04ab35a3557b9" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } } }
I20260811 17:18:53.847078 15932 sys_catalog.cc:458] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:53.848285 15929 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "feab9b4abc964873b70961c635dcbfcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } } peers { permanent_uuid: "fa9920ba049149cf8d537dd1566c64d8" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } } peers { permanent_uuid: "5ca7102b594f4fe790e04ab35a3557b9" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } } }
I20260811 17:18:53.848419 15929 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:18:53.849239 15972 mvcc.cc:205] Tried to move back new op lower bound from 7317375933733171200 to 7317375933702688768. Current Snapshot: MvccSnapshot[applied={T|T < 7317375933733171200}]
I20260811 17:18:53.849484 15932 sys_catalog.cc:455] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "feab9b4abc964873b70961c635dcbfcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } } peers { permanent_uuid: "fa9920ba049149cf8d537dd1566c64d8" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } } peers { permanent_uuid: "5ca7102b594f4fe790e04ab35a3557b9" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } } }
I20260811 17:18:53.849606 15932 sys_catalog.cc:458] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb [sys.catalog]: This master's current role is: LEADER
I20260811 17:18:53.850019 15912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "feab9b4abc964873b70961c635dcbfcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feab9b4abc964873b70961c635dcbfcb" member_type: VOTER last_known_addr { host: "127.21.54.190" port: 34033 } } peers { permanent_uuid: "fa9920ba049149cf8d537dd1566c64d8" member_type: VOTER last_known_addr { host: "127.21.54.189" port: 43061 } } peers { permanent_uuid: "5ca7102b594f4fe790e04ab35a3557b9" member_type: VOTER last_known_addr { host: "127.21.54.188" port: 44723 } } }
I20260811 17:18:53.850164 15912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:18:53.851219 15974 mvcc.cc:205] Tried to move back new op lower bound from 7317375933733171200 to 7317375933702688768. Current Snapshot: MvccSnapshot[applied={T|T < 7317375933733171200}]
I20260811 17:18:53.857964 15910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:53.813795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38854 (local address 127.21.54.190:34033)
0811 17:18:53.813901 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.813905 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.823974 (+ 10069us) server_negotiation.cc:407] Connection header received
0811 17:18:53.824158 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.824161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.824211 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.824263 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.842653 (+ 18390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.843625 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.844825 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:53.845166 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:53.845256 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:53.849457 (+ 4201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:18:53.849463 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:18:53.849465 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:18:53.849499 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:18:53.855368 (+ 5869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.856091 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.856099 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.856101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.856220 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:18:53.856651 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:18:53.856655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:18:53.856657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:18:53.857030 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:18:53.857214 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:53.857622 (+ 408us) server_negotiation.cc:299] Negotiation successful
0811 17:18:53.857779 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 17:18:53.861263 15970 catalog_manager.cc:1347] Generated new cluster ID: 2917880a30ea4b2eb4fbf7d0e9635bdb
I20260811 17:18:53.861357 15970 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:18:53.883634 15970 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:18:53.885002 15970 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:18:53.893750 15970 catalog_manager.cc:5868] T 00000000000000000000000000000000 P feab9b4abc964873b70961c635dcbfcb: Generated new TSK 0
I20260811 17:18:53.894866 15970 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 17:18:53.929858 15816 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:53.927981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46848 (local address 127.21.54.189:43061)
0811 17:18:53.928141 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.928146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.928572 (+ 426us) server_negotiation.cc:407] Connection header received
0811 17:18:53.928821 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.928824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.928882 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.928944 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.929703 (+ 759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46848: BlockingRecv error: recv got EOF from 127.0.0.1:46848 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 17:18:53.931326 15926 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:53.928957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36704 (local address 127.21.54.188:44723)
0811 17:18:53.929079 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.929083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.930287 (+ 1204us) server_negotiation.cc:407] Connection header received
0811 17:18:53.930478 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.930481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.930534 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.930600 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.931195 (+ 595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36704: BlockingRecv error: recv got EOF from 127.0.0.1:36704 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 17:18:53.931360 15910 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:53.929364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38856 (local address 127.21.54.190:34033)
0811 17:18:53.929481 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.929486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.930395 (+ 909us) server_negotiation.cc:407] Connection header received
0811 17:18:53.930579 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.930582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.930634 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.930686 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.931233 (+ 547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38856: BlockingRecv error: recv got EOF from 127.0.0.1:38856 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 17:18:53.935194 15816 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:53.933940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46862 (local address 127.21.54.189:43061)
0811 17:18:53.934045 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.934048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.934281 (+ 233us) server_negotiation.cc:407] Connection header received
0811 17:18:53.934441 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.934444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.934552 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.934637 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.935080 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46862: BlockingRecv error: recv got EOF from 127.0.0.1:46862 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 17:18:53.939443 15816 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:53.938438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46866 (local address 127.21.54.189:43061)
0811 17:18:53.938556 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.938560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.938576 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:18:53.938734 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.938737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.938794 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.938886 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.939327 (+ 441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46866: BlockingRecv error: recv got EOF from 127.0.0.1:46866 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 17:18:53.945195 15816 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:53.943912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46876 (local address 127.21.54.189:43061)
0811 17:18:53.944023 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.944027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.944296 (+ 269us) server_negotiation.cc:407] Connection header received
0811 17:18:53.944443 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.944446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.944502 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.944611 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.945064 (+ 453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46876: BlockingRecv error: recv got EOF from 127.0.0.1:46876 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 17:18:53.952879 15816 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:53.951539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46886 (local address 127.21.54.189:43061)
0811 17:18:53.951667 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.951672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.951901 (+ 229us) server_negotiation.cc:407] Connection header received
0811 17:18:53.952048 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.952052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.952116 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.952212 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.952752 (+ 540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46886: BlockingRecv error: recv got EOF from 127.0.0.1:46886 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 17:18:53.964685 15816 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:53.963140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46890 (local address 127.21.54.189:43061)
0811 17:18:53.963278 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.963283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.963587 (+ 304us) server_negotiation.cc:407] Connection header received
0811 17:18:53.963740 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.963744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.963820 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.963943 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.964489 (+ 546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46890: BlockingRecv error: recv got EOF from 127.0.0.1:46890 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 17:18:53.984640 15816 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:53.983053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46906 (local address 127.21.54.189:43061)
0811 17:18:53.983200 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:53.983205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:53.983540 (+ 335us) server_negotiation.cc:407] Connection header received
0811 17:18:53.983713 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:53.983717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:53.983784 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:53.983885 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:53.984464 (+ 579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46906: BlockingRecv error: recv got EOF from 127.0.0.1:46906 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 17:18:54.021100 15816 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:54.019424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46916 (local address 127.21.54.189:43061)
0811 17:18:54.019589 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:54.019595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:54.019924 (+ 329us) server_negotiation.cc:407] Connection header received
0811 17:18:54.020151 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:54.020159 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:54.020216 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:54.020321 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:54.020944 (+ 623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46916: BlockingRecv error: recv got EOF from 127.0.0.1:46916 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 17:18:54.089056 15816 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:54.087645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46920 (local address 127.21.54.189:43061)
0811 17:18:54.087830 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:54.087834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:54.087949 (+ 115us) server_negotiation.cc:407] Connection header received
0811 17:18:54.088144 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:54.088147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:54.088206 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:54.088307 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:54.088898 (+ 591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46920: BlockingRecv error: recv got EOF from 127.0.0.1:46920 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 17:18:54.221069 15816 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:54.219601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46926 (local address 127.21.54.189:43061)
0811 17:18:54.219775 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:54.219780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:54.219934 (+ 154us) server_negotiation.cc:407] Connection header received
0811 17:18:54.220036 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:54.220038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:54.220092 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:54.220176 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:54.220903 (+ 727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46926: BlockingRecv error: recv got EOF from 127.0.0.1:46926 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 17:18:54.481150 15816 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:18:54.479625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46932 (local address 127.21.54.189:43061)
0811 17:18:54.479804 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:54.479809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:54.480158 (+ 349us) server_negotiation.cc:407] Connection header received
0811 17:18:54.480244 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:54.480247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:54.480301 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:54.480389 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:18:54.480993 (+ 604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46932: BlockingRecv error: recv got EOF from 127.0.0.1:46932 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 17:18:54.695195 15924 catalog_manager.cc:1259] Loaded cluster ID: 2917880a30ea4b2eb4fbf7d0e9635bdb
I20260811 17:18:54.695286 15924 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9: loading cluster ID for follower catalog manager: success
I20260811 17:18:54.697225 15924 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9: acquiring CA information for follower catalog manager: success
I20260811 17:18:54.698449 15924 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5ca7102b594f4fe790e04ab35a3557b9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 17:18:54.804360 15942 catalog_manager.cc:1259] Loaded cluster ID: 2917880a30ea4b2eb4fbf7d0e9635bdb
I20260811 17:18:54.804450 15942 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8: loading cluster ID for follower catalog manager: success
I20260811 17:18:54.806232 15942 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8: acquiring CA information for follower catalog manager: success
I20260811 17:18:54.807428 15942 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fa9920ba049149cf8d537dd1566c64d8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 17:18:55.000155 16063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:54.995314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46934 (local address 127.21.54.189:43061)
0811 17:18:54.995564 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:54.995569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:54.995756 (+ 187us) server_negotiation.cc:407] Connection header received
0811 17:18:54.995940 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:54.995943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:54.996077 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:54.996254 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:18:54.996545 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:54.997072 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:54.998233 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:54.998588 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:54.998717 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:54.999097 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:18:54.999681 (+ 584us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:18:54.999830 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:54.999881 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 17:18:54.999947 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":93,"threads_started":1}
I20260811 17:18:55.007638 16073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:55.003732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36708 (local address 127.21.54.188:44723)
0811 17:18:55.003977 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:55.003981 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:55.003995 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:18:55.004118 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:55.004120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:55.004240 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:55.004341 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:18:55.004627 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:55.005158 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:55.006092 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:55.006324 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:55.006409 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:55.006735 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:18:55.007192 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:18:55.007290 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:55.007449 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 17:18:55.007501 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":105,"threads_started":1}
I20260811 17:18:55.015036 16063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:55.010656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46942 (local address 127.21.54.189:43061)
0811 17:18:55.010787 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:55.010792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:55.011151 (+ 359us) server_negotiation.cc:407] Connection header received
0811 17:18:55.011350 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:55.011354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:55.011419 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:55.011491 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:18:55.011961 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:55.012765 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:55.013849 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:55.014195 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:55.014308 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:55.014561 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:18:55.014653 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:18:55.014725 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:55.014811 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 17:18:55.014869 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 17:18:55.015199 16073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:55.011333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36712 (local address 127.21.54.188:44723)
0811 17:18:55.011435 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:55.011439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:55.011733 (+ 294us) server_negotiation.cc:407] Connection header received
0811 17:18:55.011848 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:55.011851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:55.011896 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:55.011981 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:18:55.012321 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:55.013035 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:55.013949 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:55.014277 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:55.014367 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:55.014709 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:18:55.014791 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:18:55.014866 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:55.015016 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 17:18:55.015077 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 17:18:55.015533 16079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:18:55.010651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38864 (local address 127.21.54.190:34033)
0811 17:18:55.010889 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 17:18:55.010894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:18:55.010972 (+ 78us) server_negotiation.cc:407] Connection header received
0811 17:18:55.011188 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:18:55.011190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:18:55.011351 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:18:55.011475 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:18:55.011930 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:55.012465 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:55.013668 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:18:55.014022 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:18:55.014130 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:18:55.014477 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:18:55.014843 (+ 366us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:18:55.014929 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 17:18:55.015318 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 17:18:55.015384 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":66,"threads_started":1}
I20260811 17:18:55.018627 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 15633
I20260811 17:18:55.028430 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 15726
I20260811 17:18:55.038672 21722 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2dz75L/build/debug/bin/kudu with pid 15820
2026-08-11T17:18:55Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3474 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6949 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (306939 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST